skip to main content skip to footer

Systems and Methods for Content Scoring of Spoken Responses

Author(s):
Wang, Xinhao; Zechner, Klaus; Xie, Shasha
Patent Issued:
May 16, 2017
Patent Number:
9,652,991
Source:
ETS Patent
Document Type:
Patent
Family ID:
51488246
Subject/Key Words:
Patent, Active Patent, Non-Native Speakers, Automatic Speech Recognition, Content-Based Scoring, Automated Scoring of Speech, Automated Scoring and Natural Language Processing, Spoken Language Assessment

Abstract

Computer-implemented systems and methods are provided for automatically scoring the content of moderately predictable responses. For example, a computer performing the content scoring analysis can receive a response (either in text or spoken form) to a prompt. The computer can determine the content correctness of the response by analyzing one or more content features. One of the content features is analyzed by applying one or more regular expressions, determined based on training responses associated with the prompt. Another content feature is analyzed by applying one or more context free grammars, determined based on training responses associated with the prompt. Another content feature is analyzed by applying a keyword list, determined based on the test prompt eliciting the response and/or stimulus material. Another content feature is analyzed by applying one or more probabilistic n-gram models, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a POS response vector, determined based on the response, to one or more POS training vectors, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a response n-gram count to one or more training n-gram counts using an n-gram matching evaluation metric (e.g., BLEU). Another content feature is analyzed by comparing the response to one to training responses associated with the prompt using a dissimilarity metric (e.g., edit distance and word error rate).

Read More