Computerized System for Scoring Constructed Responses and Methods for Training, Monitoring, and Evaluating Human Rater's Scoring of Constructed Responses (Expired)
- Author(s):
- Romano, Frank J.; Grant, Mark; Farnum, Marisa
- Patent Issued:
- Nov 23, 1999
- Patent Number:
- 5,991,595
- Source:
- ETS Patent
- Document Type:
- Patent
- Family ID:
- 26721397
- Subject/Key Words:
- Patent, Expired Patent, Computer Based Environment, Constructed-Response Scoring, Distributed Scoring, Human Raters, Training
Abstract
Systems and methods for presentation to raters of constructed responses to test questions in electronic workfolders. The systems and methods provide for selection and designation of constructed responses into categories for use in quality control functions and training applications. A storage device is used to relationally store data regarding the raters, the constructed responses, and the scores awarded for use in selecting constructed responses for use in quality control and training. The system also comprises rater stations and processing systems for selectively transmitting constructed responses to the rater stations and receiving scores awarded by the raters assigned to the rater stations during both training and actual production scoring. The system further comprises rater stations and processing systems for displaying rater performance data, topic performance data, constructed response volume data, and candidate sample volume data.