Systems and Methods for Generating Automated Evaluation Models
- Author(s):
- Deane, Paul; Bridgeman, Brent; Ramineni, Chaitanya
- Patent Issued:
- Oct 15, 2019
- Patent Number:
- 10,446,044
- Source:
- ETS Patent
- Document Type:
- Patent
- Family ID:
- 52019529
- Subject/Key Words:
- Patent, Active Patent, Automated Scoring and Natural Language Processing, Scoring Models, Machine Learning, Constructed-Response Scoring, Automated Writing Evaluation (AWE)
Abstract
A computer-implemented method of calibrating an assessment model for assessing responses includes accessing a plurality of training responses with a processing system for training an assessment model. The processing system analyzes the plurality of training responses to derive values of multiple features of the training responses. The processing system trains the assessment model based on the values of the multiple features of the training responses and a portfolio score for each individual associated with the plurality of training responses utilized in the training. The portfolio score for each individual corresponds to a measure of proficiency based on multiple writing samples constructed by the individual. The processing system determines, based on said training, a weight for each of the multiple features. The processing system calibrates the assessment model to include the weights for at least some of the features such that the assessment model is configured to generate scores for responses.