NAEP Summer Intern Blog Series: Introduction to Contextualized Assessments

December 8, 2021

Throughout the summer, we’re highlighting the experiences of some of our NAEP interns, in their own words. 

This week’s post is from the perspective of Margaret Wong. This summer, Margaret is collaborating with ETS senior product management lead and mentor, Jan Alegre, on a project exploring innovations in reporting NAEP survey questionnaire data: 

This summer, I have been working on a project with Dr. Jan Alegre as my mentor developing innovative ways of reporting NAEP questionnaire data on the nationsreportcard.gov website. It has been incredibly interesting to comb through the usability reports on the design of NAEP reporting projects in order to improve it. The seminar this week, titled “Introduction to Contextualized Assessments”, was presented by ETSers Robert Finnegan, Chris Agard, and Madeleine Keehner. It provided even more insight into the importance of contextual data in evaluating student performance factors. 

Listening to the seminar, I was particularly excited to learn more about contextual information being collected from students through student survey questionnaires (SQ). Contextual factors include a student’s “Opportunity to Learn” and non-cognitive factors. Opportunity to Learn, or OTL, are factors related to whether a student is exposed to opportunities to acquire knowledge, in or out of school. Non-cognitive factors refer to the skills, strategies, attitudes, and behaviors that are distinct from content knowledge and academic skills. For example, an OTL such as access to books may impact how much reading a student is able to independently do, which may have an effect on reading performance. On the other hand, a non-cognitive factor could be a student’s personal motivation to read books outside the classroom. If a student is less intrinsically motivated to read, their reading scores on NAEP assessments may be lower. 

With the sheer amount of data coming out of the survey questionnaires, there comes a need to report it in a digestible and meaningful way for interested stakeholders. That is why I’m so excited to be working with Dr. Jan Alegre this summer to investigate ways to improve the reporting of Survey Questionnaires on the NAEP website, so audiences of various backgrounds can learn more about these factors influencing student performance. So far, I have been entrenched in the user feedback on the reporting of these survey questionnaire results. Unsurprisingly, audiences of various backgrounds in education have been extremely interested in this data. I am excited to start interviewing stakeholders to develop innovative dashboards for reporting SQ data. 

Margaret Wong is a NAEP-Summer Undergraduate Research Experience (SURE) intern who recently completed her undergraduate degree in Human Systems Engineering from Arizona State University and is currently working toward her master’s degree in the same discipline also at Arizona State University.