skip to main content skip to footer

Network Analysis of Conversation Data for Engineering Professional Skills Assessment SNA

Author(s):
Zhu, Mengxiao; Zhang, Mo
Publication Year:
2017
Report Number:
RR-17-59
Source:
ETS Research Report
Document Type:
Report
Page Count:
15
Subject/Key Words:
Conversation Analysis, Engineering Education, Collaborative Assessment, Problem Solving, Group Discussion, Social Network Analysis (SNA), Workplace Skills, Higher Education

Abstract

In this paper, we examine the student group discussion processes in a scenario-based assessment of engineering professional skills called Engineering Professional Skills Assessment (EPSA). In the assessment, the students were evaluated through a discussion on a scenario related to an engineering problem with no clear-cut solution. We applied various social network analysis (SNA) techniques and constructed 2 types of networks to model the patterns of the communication process in the student discussion and the connections among engineering professional skills (EPS) during the discussion process. We found that some network statistics were statistically related to team scores (or performance), such as the density of the EPS networks. We also investigated the temporal pattern of the discussion process by comparing the network statistics across 3 different time points. Results showed that the middle and end of the discussion processes differed significantly on 2 network measures: communication density and EPS connectedness. Results of the study have implications for instruction and scoring of such performance tasks. Limitations and next steps are also discussed.

Read More