Tying Machine Learning to Next Generation Science Standards
May 11, 2021
The Next Generation Science Standards (NGSS®) call for multidimensional learning that stresses the integration of scientific practices with conceptual understanding of core ideas to better prepare students for today’s world. The most important aspect of implementing NGSS in the classroom is its emphasis on multidimensional science learning and engaging teacher practice that supports students. As students navigate life each day, they naturally develop intuitive knowledge and reasoning patterns that shape their learning including science learning. As students engage in these experiences, it’s equally important that teachers are equipped to interpret students’ written responses shaped by these experiences. A result of this need has taken the form of a recent National Science Foundation funded study, Student Reasoning Patterns in Next Generation Science Standards Assessment (SPIN-NGSS), I lead alongside my ETS colleagues aimed at developing automated tools. These tools are intended to help teachers interpret data from assessments aligned to these standards in order to reveal student reasoning patterns, helping to reflect particular weaknesses in student reasoning.
Building off what students bring in class
Students bring diverse ideas, reasoning skills and life experiences to the classroom. When it comes to understanding the world of science, early childhood experiences may have led children to develop causal reasoning skills that they apply broadly. Building on students’ everyday experiences can be an effective way to expand upon existing ideas and reasoning strategies children may have developed. Students’ reasoning patterns reflect a diversity of intuitive ideas and can be considered stepping-stones toward sophisticated scientific understanding. However, students may sometimes bring ideas to the classroom that the teacher considers inaccurate and are later instructed with accurate concepts. This replacement strategy may result in students memorizing school knowledge but falling back on their misconceptions when they are asked to explain scientific phenomena.
Student reasoning patterns in next generation science learning and assessments
Research has identified distinct “styles of reasoning” of scientists that involve the three dimensions of knowledge required by the NGSS namely: the disciplinary core ideas (DCIs), science and engineering practices (SEPs) and crosscutting concepts (CCCs). There is less documentation of student reasoning patterns in multidimensional learning. With more NGSS-aligned assessments available, there are opportunities to conduct research on characteristic features in student reasoning.
As part of our grant project, my ETS colleagues and I have been utilizing existing NGSS-aligned assessment data to identify typical student reasoning patterns. For example, when explaining a science-specific concept, we found that some students focused on describing observations and data only, while others only provided scientific principles without referring to data or evidence. Finally, we found some students attempted to integrate both data and scientific principles into their reasoning. Diagnosing these reasoning patterns is useful for generating personalized feedback to address gaps in student reasoning. In science classrooms, teachers need help to identify student reasoning patterns.
Automated diagnosis of student reasoning patterns
To help teachers better attend to student ways of reasoning, our team has developed machine learning models to automate the diagnosis of student reasoning patterns based on key features related to NGSS dimensions. These models provide both a reasoning pattern label and evidence in student responses associated with the pattern. As part of this process, content experts first coded a set of student responses. Then Natural Language Processing (NLP) experts used the human codes to train computers to develop automated models. A two-stage classification approach was applied. First phase classification identifies parts of responses related to the NGSS dimensions. The second classifier automatically classifies entire responses with a reasoning pattern. The team is continuing to validate our models and designing an automated feedback tool to scaffold multidimensional learning in science classrooms. SPIN-NGSS fills the gap of diagnosing student reasoning patterns by tying machine learning with NGSS learning. The products of SPIN-NGSS have the potential to enhance teachers’ use of science assessments to facilitate student learning through individualized and immediate feedback.
Lei Liu is a Managing Senior Research Scientist at ETS and the Principal Investigator of the SPIN-NGSS NSF grant. Co-PIs of the SPIN-NGSS grant project are Dante Cisterna (Associate Research Developer, ETS), Aoife Cahill (Managing Senior Research Scientist, ETS), and Matthew Johnson (Principal Research Director, ETS).
View the project team's video that highlights their accomplishments which will appeared at the 2021 STEM for All Video Showcase.
This material is based upon work supported by the National Science Foundation under Grant #2000492. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.