skip to main content skip to footer

A Preliminary Analysis of Keystroke Log Data From a Timed Writing Task CBAL

Author(s):
Almond, Russell G.; Deane, Paul; Quinlan, Thomas; Wagner, Michael; Sydorenko, Tetyana
Publication Year:
2012
Report Number:
RR-12-23
Source:
ETS Research Report
Document Type:
Report
Page Count:
61
Subject/Key Words:
Writing Assessment, Writing Fluency, Automated Scoring, Timing, Mixture Models, Cognitively Based Assessment of, for, and as Learning (CBAL)

Abstract

The Fall 2007 and Spring 2008 pilot tests for the CBAL™ Writing assessment included experimental keystroke logging capabilities. This report documents the approaches used to capture the keystroke logs and the algorithms used to process the outputs. It also includes some preliminary findings based on the pilot data. In particular, it notes that the distribution of most of the pause length is consistent with data generated from a mixture of lognormal distributions. This corresponds to a cognitive model in which some pauses are merely part of the transcription (i.e., typing) process and some are part of more involved cognitive process (e.g., attention to writing conventions, word choice, and planning). In the pilot data, many of the features extracted from the keystroke logs were correlated with human scores. Due to the small sample sizes of the pilot studies, these findings are suggestive, not conclusive; however, they suggest a line of analysis for a large sample containing keystroke logging gathered in the fall of 2009.

Read More