Systems and Methods for Determining Lexical Associations among Words in a Corpus
- Author(s):
- Flor, Michael; Beigman Klebanov, Beata
- Patent Issued:
- Dec 13, 2016
- Patent Number:
- 9,519,634
- Source:
- ETS Patent
- Document Type:
- Patent
- Family ID:
- 54701945
- Subject/Key Words:
- Patent, Active Patent, Lexical Tightness (LT), Corpus Analysis, Automated Scoring and Natural Language Processing
Abstract
Systems and methods are provided for identifying one or more target words of a corpus that have a lexical relationship to a plurality of provided cue words. The cue words and statistical lexical information derived from a corpus of documents are analyzed to determine candidate words that have a lexical association with the cue words. The statistical information includes numerical values indicative of probabilities of word pairs appearing together as adjacent words in a well-formed text or appearing together within a paragraph of a well-formed text. For each candidate word, a statistical association score between the candidate word and each of the cue words is determined. An aggregate score for each of the candidate words is determined based on the statistical association scores. One or more of the candidate words are selected to be the one or more target words based on the aggregate scores.