Video duration: 2:37
On-screen: [ETS® R&D Connections, No. 26. Simulations of Thought: The Role of Computational Cognitive Models in Assessment.]
Speaker: Jung Aa Moon, Associate Research Scientist, ETS - My name is Jung Aa Moon. I am a researcher in ETS's Cognitive and Technology Sciences Center in the Research & Development division.
Speaker: Michelle LaMar, Research Scientist, ETS - My name is Michelle LaMar. I am also a researcher in ETS's Research & Development division.
On-screen: [Michelle LaMar, Research Scientist, ETS; Bridgid Finn, Managing Senior Research Scientist, ETS; Irvin R. Katz, Senior Research Director, ETS; Jung Aa Moon, Associate Research Scientist, ETS]
Speaker: Jung Aa Moon - We have written an article for R&D Connections together with our colleagues, Irvin Katz and Bridgid Finn. The article exemplifies how researchers at ETS use computational cognitive models within three areas of research:
- assessment development,
- evaluation of validity evidence, and
- training of raters who score test responses.
Speaker: Michelle LaMar - Computational cognitive models are computer programs that simulate mental processes – how people think. Many such models have been developed around the world. Researchers at ETS R&D use some of them in our work.
Speaker: Jung Aa Moon - For example, we have used computational cognitive models to analyze how the way a test item is designed can affect the answer, and how test takers navigate through various steps of a task. We have also used them to model how people learn and forget what they have learned. Based on this research we have come up with ideas for more efficient ways of training human raters, who may forget part of what they learned if too much time passes between training sessions.
Speaker: Michelle LaMar - Computational cognitive models also form the basis for automated detectors of strategies that test takers may use. The examples we discuss in our essay indicate that computational cognitive modeling has a great potential. The knowledge and insights it offers can lead to better understanding of students' performance and improved validity evidence. It can influence assessment development, give us more and better information about test takers, and make human scoring more efficient.
Speaker: Jung Aa Moon - We hope you enjoy reading "Simulations of Thought: The Role of Computational Cognitive Models in Assessment."
On-screen: [R&D Connections. ets.org/research. ETS® Copyright © 2018 by Educational Testing Service. All rights reserved. ETS and the ETS logo are registered trademarks of Educational Testing Service (ETS).]
Speaker: Narrator - R&D Connections is a free publication, which you can download at no cost by visiting the Research section of ETS.ORG.