Systems and Methods for Detecting Unusually Frequent Exactly Matching and Nearly Matching Test Responses
- Author(s):
- Haberman, Shelby J.; Lee, Yi-Hsuan; Papierman, Paul; Zhou, Yuxing; Subhedar, Ruchi
- Patent Issued:
- Jul 26, 2022
- Patent Number:
- 11,398,161
- Source:
- ETS Patent
- Document Type:
- Patent
- Family ID:
- 82483937
- Subject/Key Words:
- Patent, Active Patent, Automated Scoring, Test Keys, Response Analysis, Discrepancy Resolution, Machine Scoring
Abstract
Computer-implemented systems and methods are described herein for detecting unusually frequent exactly matching and nearly matching test responses. A plurality of test responses for each of a plurality of test takers is received. Evidence of a circulated key from the plurality of test responses is identified by detecting two or more test takers with exactly matching test responses for a plurality of questions. The strength of the evidence of the circulated key is analyzed by determining the probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability. Test takers with nearly matching test responses to the circulated key are then identified. All test takers with exactly matching and nearly matching test responses are tagged for further investigation.