skip to main content skip to footer

CPS-Rater: Automated Sequential Annotation for Conversations in Collaborative Problem-Solving Activities CPS

Hao, Jiangang; Chen, Lei; Flor, Michael; Liu, Lei; von Davier, Alina A.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Automated Assessment Technology, Annotation, Sequential Approach, Conversation Analysis, Random Variables, Collaboration, Science Assessment, Simulation Exercises, Collaborative Problem Solving (CPS)


Conversations in collaborative problem-solving activities can be used to probe the collaboration skills of the team members. Annotating the conversations into different collaboration skills by human raters is laborious and time consuming. In this report, we report our work on developing an automated annotation system, CPS-rater, for conversational data from collaborative activities. The linear chain conditional random field method is used to model the sequential dependencies between the turns of the conversations, and the resulting automated annotation system outperforms those systems that do not model the sequential dependency.

Read More