Automated Scoring and Natural Language Processing — Bibliography
This page contains a comprehensive list of our research publications on the use of natural language processing technology to create applications that evaluate and score test responses or for use as educational tools.
- 2012
-
- Measuring the Use of Factual Information in Test-Taker Essays
Authors: B. Beigman-Klebanov and D. Higgins (2012)
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA)
Publisher: Association for Computational Linguistics - Building Subjectivity Lexicon(s) from Scratch for Essay Data
Authors: B. Beigman-Klebanov, J. Burstein, N. Madnani, A. Faulkner, & J. Tetreault (2012)
Lecture Notes in Computer Science, 2012, Volume 7181, Computational Linguistics and Intelligent Text Processing, pp. 591–602 - Item Generation: Implications for a Validity Argument
Author: I. I. Bejar (2012)
In M. Gierl & S. Haladyna (Eds.), Automated Item Generation: Theory and Practice, pp. 40–56
Publisher: Routledge - Rater cognition: Implications for validity
Author: I. I. Bejar (2012)
Article in Educational Measurement: Issues and Practice, 31(3), 2–9. - Comparison of Human and Machine Scoring of Essays: Differences by Gender, Ethnicity, and Country
Authors: B. Bridgeman, C. Trapani, & Y. Attali (2012)
Article in Applied Measurement in Education, Vol. 25, No. 1, pp. 27–40 - Fostering Best Practices in Writing Instruction and Assessment with E-rater
Author: J. Burstein (2012)
In N. Elliott and L. Perelman (Eds.), Writing Assessment in the 21st Century: Essays in Honor of Edward M. White. (2012)
Publisher: Hampton Press - Automated Essay Scoring and Evaluation
Author: J. Burstein (2012)
In Carol Chapelle (Ed.), The Encyclopedia of Applied Linguistics.
Publisher: Wiley-Blackwell - Utilizing Cumulative Logit Model and Human Computation on Automated Speech Assessment
Author: L. Chen (2012)
In Proceedings of the 7th Workshop on the Innovative Use of NLP for Building Educational Applications, pp. 73–79
Publisher: Association for Computational Linguistics - Application of Structural Events Detected on ASR Outputs for Automated Speaking Assessment
Authors: L. Chen & S. Yoon (2012)
In Proceedings of the 13th Annual Conference of the International Communication Association, Portland, Oregon - Rethinking K–12 Writing Assessment
Author: D. Deane (2012)
In N. Elliott and L. Perelman (Eds.), Writing Assessment in the 21st Century: Essays in Honor of Edward M. White, pp. 87–100
Publisher: Hampton Press - Automatic Detection of [th] Pronunciation Errors for Chinese Learners of English
Authors: K. Evanini and B. Huang (2012)
Proceedings of the 2012 International Symposium on Automatic Detection of Errors in Pronunciation Training (ISADEPT)
Publisher: KTH, Computer Science and Communication, Department of Speech, Music and Hearing, Stockholm, Sweden - On Using Context for Automatic Correction of Non-Word Misspellings in Student Essays
Authors: M. Flor and Y. Futagi (2012)
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA)
Publisher: Association for Computational Linguistics - Difficulty Modeling and Automatic Generation of Quantitative Items: Recent Advances and Possible Next Steps
Authors: E. A. Graf & J. H. Fife (2012)
In M. Gierl & T. Haladyna (Eds.), Automatic Item Generation: Theory and Practice, pp. 157–180
Publisher: Routledge - Discriminative Edit Models for Paraphrase Scoring
Authors: M. Heilman and N. Madnani (2012)
Proceedings of the 6th International Workshop of Semantic Evaluation (SemEval)
Publisher: Association for Computational Linguistics - Precision Isn't Everything: A Hybrid Approach to Grammatical Error Detection
Authors: M. Heilman, A. Cahill, and J. Tetreault (2012)
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA)
Publisher: Association for Computational Linguistics - Correcting Comma Errors in Learner Essays, and Restoring Commas in Newswire Text
Authors: R. Israel, J. Tetreault, and M. Chodorow (2012)
Proceedings of the 12th Annual NAACL-HLT Conference
Publisher: Association for Computational Linguistics - Exploring Grammatical Error Correction with Not-So-Crummy Machine Translation
Authors: N. Madnani, J. Tetreault, and M. Chodorow (2012)
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA)
Publisher: Association for Computational Linguistics - Re-examining Machine Translation Metrics for Paraphrase Identification
Authors: N. Madnani, J. Tetreault, and M. Chodorow (2012)
Proceedings of the 2012 Meeting of the North American Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
Publisher: Association for Computational Linguistics - Identifying High Level Organizational Elements in Argumentative Discourse
Authors: N. Madnani, M. Heilman, J. Tetreault, and M. Chodorow (2012)
Proceedings of the 2012 Meeting of the North American Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
Publisher: Association for Computational Linguistics - A Framework for Evaluation and Use of Automated Scoring
Authors: D. M. Williamson, X. Xi, & F. J. Breyer (2012)
Article in Educational Measurement: Issues and Practices, Vol. 31, No. 1, pp. 2–13 - The Conceptual and Scientific Basis for Automated Scoring of Performance Items
Author: D. M. Williamson (2012)
In R. W. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predictions for the future, pp. 157–193
Publisher: Information Age Publishing - A Comparison of Two Scoring Methods for an Automated Speech Scoring System
Authors: X. Xi, D. Higgins, K. Zechner, and D. Williamson (2012)
Language Testing, Vol. 29, No. 3, pp. 371–394 - Validity and the Automated Scoring of Performance Tests
Author: X. Xi
In G. Fulcher & F. Davidson (Eds.), The Routledge Handbook of Language Testing, pp. 438–451
Publisher: Routledge - Exploring Content Features for Automated Speech Scoring
Authors: S. Xie, K. Evanini, and K. Zechner
Proceedings of the 2012 Meeting of the North American Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
Publisher: Association for Computational Linguistics - Vocabulary Profiles as a Measure of Vocabulary Sophistication
Authors: S. Yoon, S. Bhat, and K. Zechner (2012)
Proceedings of the 7th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications (BEA)
Publisher: Association for Computational Linguistics - Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures
Authors: S. Yoon & S. Bhat (2012)
In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)
Publisher: Association for Computational Linguistics - Using Automatic Speech Recognition to Assess the Reading Proficiency of a Diverse Sample of Middle School Students
Authors: K. Zechner, K. Evanini, & C. Laitusis (2012)
In Proceedings of the Interspeech Workshop on Child, Computer, and Interaction
Publisher: International Speech Communication Association
- Measuring the Use of Factual Information in Test-Taker Essays
- 2010–2011
-
- Automated Subscores for TOEFL iBT Independent Essays
Author: Y. Attali (2011)
ETS Research Report No. RR-11-39 - A Differential Word Use Measure for Content Analysis in Automated Essay Scoring
Author: Y. Attali (2011)
ETS Research Report No. RR-11-36 - A Validity-Based Approach to Quality Control and Assurance of Automated Scoring
Author: I.I. Bejar (2011)
Article in Assessment in Education: Principles, Policy & Practice, Vol. 18, No. 3, pp. 319–341 - Automated Scoring of Constructed-Response Literacy and Mathematics Items
Author: R. E. Bennett (2011)
Publisher: White paper published by Arabella Philanthropic Investment Advisors - Semantic Role Labeling Without Treebanks?
Authors: S. A. Boxwell, C. Brew, J. Baldridge, D. N. Mehay, & S. Ravi (2011)
Paper in Proceedings of the 5th International Joint Conference on Natural Language Processing, pp. 192–200
Publisher: Asian Federation of Natural Language Processing - Detecting Structural Events for Assessing Non-Native Speech
Authors: L. Chen & S.-Y. Yoon (2011)
Paper in Proceedings of the ACL-HLT-2011 Workshop on Innovative Use of NLP for Building Educational Applications, pp. 38–45
Publisher: Association for Computational Linguistics - Applying Rhythm Features to Automatically Assess Non-Native Speech
Authors: L. Chen & K. Zechner (2011)
Paper in Proceedings of Interspeech
Publisher: International Speech Communication Association - Computing & Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech
Authors: M. Chen & K. Zechner (2011)
Paper in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics & the Human Language Technologies Conference (ACL-HLT-2011)
Publisher: Association for Computational Linguistics - Writing Assessment and Cognition
Author: P. Deane (2011)
ETS Research Report No. RR-11-14 - Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency
Authors: P. Deane, T. Quinlan, & I. Kostin (2011)
ETS Research Report No. RR-11-16 - The CBAL Summative Writing Assessment: A Draft 8th-Grade Design
Authors: P. Deane, M. Fowles, D. Baldwin, & H. Persky (2011)
ETS Research Memorandum RM-11-01 - The Importance of Optimal Parameter Setting for Pitch Extraction
Authors: K. Evanini, C. Lai, & K. Zechner (2011)
Paper in Proceedings of Meetings on Acoustics, Vol. 11
Publisher: Acoustical Society of America - Using Crowdsourcing to Provide Prosodic Annotations for Non-Native Speech
Author: K.Evanini & K. Zechner (2011)
Paper in Proceedings of Interspeech
Publisher: International Speech Communication Association - Use of e-rater in Scoring of the TOEFL iBT Writing Test
Author: S.J. Haberman (2011)
ETS Research Report No. RR-11-25 - A Three-Stage Approach to the Automated Scoring of Spontaneous Spoken Responses
Authors: D. Higgins, K. Zechner, X. Xi, & D. Williamson (2011)
Article in Computer Speech & Language, Vol. 25, No. 2, pp. 282–306 - Applying Feature Bagging for More Accurate and Robust Automated Speaking Assessment
Author: L. Chen (2011)
Paper in Proceedings of the 2011 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 473–477
Publisher: IEEE - Performance of a Generic Approach in Automated Essay Scoring
Authors: Y. Attali, B. Bridgeman & C. Trapani (2010)
Article in Journal of Technology, Learning & Assessment, Vol. 10, No. 3 - The Application of the Cumulative Logistic Regression Model to Automated Essay Scoring
Authors: S. J. Haberman & S. Sinharay (2010)
Article in Journal of Educational & Behavioral Statistics, Vol. 35, No. 5, pp. 586–602 - Application of Evidence-Centered Assessment Design to the Advanced Placement Redesign: A Graphic Restatement
Author: I. I. Bejar (2010)
Article in Applied Measurement in Education, Vol. 23, pp. 378–391 - Progress and New Directions in Technology for Automated Essay Evaluation
Authors: J. Burstein & M. Chodorow (2010)
Chapter in The Oxford Handbook of Applied Linguistics, 2nd Edition, pp. 487–497
Editor: R. Kaplan
Publisher: Oxford University Press - Using Entity-Based Features to Model Coherence in Student Essays
Authors: J. Burstein, J. Tetreault, & S. Andreyev (2010)
Paper in Human language technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pp. 681–684
Publisher: Association for Computational Linguistics - Towards Using Structural Events to Assess Non-Native Speech
Authors: L. Chen, J. Tetreault, & X. Xi (2010)
Paper in NAACL-HLT 2010 Proceedings of the 5th Workshop on Building Educational Applications (BEA-5) - The Utility of Article and Preposition Error Correction Systems for English Language Learners: Feedback and Assessment
Authors: M. Chodorow, M. Gamon, & J. Tetreault (2010)
Article in Language Testing, Vol. 27, No. 3, pp. 419–436 - Using Amazon Mechanical Turk for Transcription of Non-Native Speech
Authors: K. Evanini, D. Higgins, & K. Zechner (2010)
Paper in Proceedings of the Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, HLT-NAACL 2010
Publisher: Association for Computational Linguistics - Generating Automated Text Complexity Classifications That Are Aligned with Targeted Text Complexity Standards
Authors: K. M. Sheehan, I. Kostin, Y. Futagi, & M. Flor (2010)
ETS Research Report No. RR-10-28 - Using an Error-Annotated Learned Corpus to Develop an ESL/EFL Error Correction System
Authors: N.-R. Han, J. Tetreault, S.-H. Lee, & J.-Y. Ha (2010)
Paper in Proceedings of the International Conference on Language Resources and Evaluation (LREC 2010)
Publisher: European Language Resources Association - The Web is not a Person, Berners-Lee is not an ORGANIZATION and African-Americans are not LOCATIONS: Analysis of the Performance of Named-Entity Recognition
Authors: R. Krovetz, P. Deane, & N. Madnani (2011)
Proceedings of the Workshop on Multiword Expressions: from Parsing and Generation to the Real World (MWE 2011), pp. 57–64
Publisher: Association for Computational Linguistics - Automated Grammatical Error Detection for Language Learners
Authors: C. Leacock, M. Chodorow, M. Gamon, & J. Tetreault (2010)
Monograph in Synthesis Lectures on Human Language Technologies
Publisher: Morgan & Claypool - Unsupervised Prompt Expansion for Off-Topic Essay Detection
Authors: A. Louis & D. Higgins (2010)
Paper in Proceedings of the Workshop on Building Educational Applications, HLT-NAACL 2010
Publisher: Association for Computational Linguistics - iBLEU: Interactively Scoring and Debugging Statistical Machine Translation Systems
Author: N. Madnani (2011)
Paper in Proceedings of the Fifth IEEE International Conference on Semantic Computing (Demos)
Publisher: IEEE Computer Society - They Can Help: Using Crowdsourcing to Improve the Evaluation of Grammatical Error Detection Systems
Authors: N. Madnani, J. Tetreault, M. Chodorow, & A. Rozovskaya (2011)
Paper in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Shortpapers, pp. 508–513
Publisher: Association for Computational Linguistics - Technology Supports for Assessment Design
Authors: R.J. Mislevy, F. Winters, I.I. Bejar, R.E. Bennett & G.D. Haertel (2010)
Chapter in International Encyclopedia of Education, Vol. 8, 3rd ed., pp. 56–65
Publisher: Elsevier - E-rating Machine Translation
Authors: K. Parton, J. Tetreault, N. Madnani, & M. Chodorow (2011)
Paper in Proceedings of the 6th Workshop on Statistical Machine Translation, pp. 108–115
Publisher: Association for Computational Linguistics - Scoring the TOEFL® Independent Essay Automatically: Reactions of Test Takers & Test Score Users
Author: D. E. Powers, A. H. Cumming & R. Kanton
ETS Research Memorandum No. RM-11-34, TOEFL No. MS-22 - Automated Essay Scoring: Writing Assessment and Instruction
Authors: M. D. Shermis, J. Burstein, D. Higgins, & K. Zechner (2010)
Article in International Encyclopedia of Education, 3rd edition, Vol. 4, pp. 20–26
Editors: E. Baker, B. McGaw, & N. S. Petersen
Publisher: Elsevier - Maximum Entropy for the Automatic Content Scoring of Free-Text Responses
Author: J. Z. Sukkarieh (2010)
In Proceedings of MaxEnt 2010 - Building a Textual Entailment Suite for Evaluating Content Scoring Technologies
Authors: J. Z. Sukkarieh & E. Bolge (2010)
Paper in Proceedings of the Seventh Conference on International Language Resources and Evaluation, pp. 3149–3156
Publisher: European Language Resources Association - Gaussian Mixture Modeling of Vowel Durations for Automated Assessment of Non-native Speech
Authors: X. Sun & K. Evanini (2011)
Paper in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 5716–5719
Publisher: The Institute of Electrical and Electronics Engineers - Using Parse Features for Preposition Selection and Error Detection
Authors: J. Tetreault, J. Foster, & M. Chodorow (2010)
Paper in Proceedings of the 2010 Association for Computational Linguistics (ACL 2010)
Publisher: Association for Computational Linguistics - Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk
Authors: J. Tetreault, E. Filatova, & M. Chodorow (2010)
Paper in NAACL-HLT: 2010 Proceedings of the 5th Workshop on Building Educational Applications (BEA-5)
Publisher: Association for Computational Linguistics - Automated Scoring for the Assessment of Common Core Standards
Authors: D. M. Williamson et. al. (2010)
White Paper
Publisher: Educational Testing Service - Exploiting Syntactic and Distributional Information for Spelling Correction with Web-Scale N-gram Models
Authors: W. Xu, J. Tetreault, M. Chodorow, R. Grishman, & L. Zhao (2011)
Paper in Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 1291–1300
Publisher: Association for Computational Linguistics - Predicting Word Accuracy for the Automatic Speech Recognition of Non-Native Speech
Authors: S. Yoon, L. Chen & K. Zechner (2010)
Paper in Proceedings of Interspeech, pp. 773–776
Publisher: International Speech Communication Association - Non-scorable Response Detection for Automated Speaking Proficiency Assessment
Authors: S.-Y. Yoon, K. Evanini, & K. Zechner (2011)
Paper in Proceedings of the ACL-HLT 6th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 152–160
Publisher: Association for Computational Linguistics - Landmark-based Automated Pronunciation Error Detection
Authors: S. Yoon, M. Hasegawa-Johnson & R. Sproat (2010)
Paper in Proceedings of Interspeech, pp. 614–617
Publisher: International Speech Communication Association - Non-English Response Detection Method for Automated Proficiency Scoring System
Authors: S. Yoon & D. Higgins (2011)
Paper in Proceedings of the ACL-HLT-2011 Workshop on Innovative Use of NLP for Building Educational Applications, pp. 161–169
Publisher: Association for Computational Linguistics - Automated Scoring of CBAL Mathematics Tasks with m-rater
Author: J. Fife (September 2011)
ETS Research Memorandum ETS RM–11-12
- Automated Subscores for TOEFL iBT Independent Essays
- 2008–09
-
- Opportunities for Natural Language Processing in Education
Author: J. Burstein (2009)
Chapter in Springer Lecture Notes in Computer Science, Vol. 5449, pp. 6–27
Editor: A. Gebulkh
Publisher: Springer - Human Evaluation of Article and Noun Number Usage: Influences of Context and Construction Variability
Authors: J. Lee, J. Tetreault, & M. Chodorow (2009)
Paper in ACL 2009 Proceedings of the Linguistic Annotation Workshop III (LAW3)
Publisher: Association for Computational Linguistics - When Do Standard Approaches for Measuring Vocabulary Difficulty, Syntactic Complexity and Referential Cohesion Yield Biased Estimates of Text Difficulty?
Authors: K. M. Sheehan, I. Kostin, & Y. Futagi (2009)
Paper in Proceedings of the 30th Annual Meeting of the Cognitive Science Society - Automating Model Building in c-rater
Authors: J. Z. Sukkarieh & S. Stoyanchev (2009)
Paper in Proceedings of TextInfer: The ACL/IJCNLP 2009 Workshop on Applied Textual Inference, pp. 61–69
Editors: R. Barzilay, J.-S. Chang, & C. Sauper
Publisher: Association for Computational Linguistics - Towards Agile and Test-Driven Development in NLP Applications
Authors: J. Z. Sukkarieh & J. Kamal (2009)
Paper in Proceedings of the NAACL HLT Workshop on Software Engineering, Testing, and Quality Assurance for Natural Language Processing, pp. 42–44
Publisher: Association for Computational Linguistics - c-rater: Automatic Content Scoring for Short Constructed Responses
Authors: J. Z. Sukkarieh & J. Blackmore (2009)
Paper in Proceedings of the 22nd International FLAIRS Conference
Publisher: Association for the Advancement of Artificial Intelligence - Examining the Use of Region Web Counts for ESL Error Detection
Authors: J. Tetreault & M. Chodorow (2009)
Paper in SEPLN 2009: Proceedings of the Web as Corpus Workshop (WAC-5)
Publisher: Association for Computational Linguistics - Automatic Scoring of Non-Native Spontaneous Speech in Tests of Spoken English
Authors: K. Zechner, D. Higgins, X. Xi, & D. Williamson (2009)
Article in Speech Communication, Vol. 51, No. 10, pp. 883–895 - Adapting the Acoustic Model of a Speech Recognizer for Varied Proficiency Non-Native Spontaneous Speech Using Read Speech with Language-Specific Pronunciation Difficulty
Authors: K. Zechner, D. Higgins, R. Lawless, Y. Futagi, S. Ohls, & G. Ivanov (2009)
Paper in Proceedings of InterSpeech-2009
Publisher: International Speech Communication Association - What did they actually say? Agreement and Disagreement among Transcribers of Non-Native Spontaneous Speech Responses in an English Proficiency Test
Author: K. Zechner (2009)
Paper in Proceedings of the International Speech Communication Association (ISCA) Spoken Language Technologies in Education (SLaTE) 2009 Workshop
Publisher: International Speech Communication Association - Automatic Scoring of Children's Read-Aloud Text Passages and Word Lists
Authors: K. Zechner, J. Sabatini, & L. Chen (2009)
Paper in Proceedings of the North American Chapter of the Association of Computational Linguistics Human Language Technologies (NAACL-HLT) Workshop on Innovative Use of NLP for Building Educational Applications
Publisher: Association for Computational Linguistics - Audio Quality Issue for Automatic Speech Assessment
Author: L. Chen (2009)
Paper in Proceedings of 2009 ISCA Workshop on Speech and Language Technology in Education (SLaTE)
Publisher: International Speech Communication Association - Improved Pronunciation Features for Construct-driven Assessment of Non-native Spontaneous Speech
Authors: L. Chen, K. Zechner, & X. Xi (2009)
Paper in NAACL '09: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics,
pp. 442–449
Publisher: Association for Computational Linguistics - Effect of Immediate Feedback and Revision on Psychometric Properties of Open-Ended GRE® Subject Test Items
Authors: Y. Attali & D. Powers (2008)
ETS Research Report No. RR-08-21 - Automated Scoring of Short-Answer Open-Ended GRE Subject Test Items- ETS Research Report
Authors: Y. Attali, D. E. Powers, M. Freedman, M. Harrison & S.A. Obetz (2008)
ETS Research Report No. RR-08-02; GRE Board Research Report No. GRE RR-04-02 - A Developmental Writing Scale
Authors: Y. Attali & D. Powers (2008)
ETS Research Report No. RR-08-19 - A Computational Approach to Detecting Collocation Errors in the Writing of Non-native Speakers of English
Authors: Y. Futagi, P. Deane, M. Chodorow, & J. Tetreault (2008)
Article in Computer Assisted Language Learning, Vol. 21, pp. 353–367 - How Important is Content in the Ratings of Essay Assessments?
Authors: M.D. Shermis, A. Shneyderman & Y. Attali (2008)
Article in Assessment in Education: Principles, Policy & Practices, Vol. 15, No. 1, pp. 91–105 - Leveraging c-rater's Automated Scoring Capability for Providing Instructional Feedback for Short Constructed Responses
Authors: J. Z. Sukkarieh & E. Bolge (2008)
Paper in Lecture Notes in Computer Science: Vol. 5091. Proceedings of the 9th International Conference on Intelligent Tutoring Systems, ITS 2008, pp. 779–783
Editors: B. P. Woolf, E. Aimeur, R. Nkambou, & S. Lajoie
Publisher: Springer-Verlag - The Ups and Downs of Prepositional Error Detection in ESL Writing
Authors: J. Tetreault & M. Chodorow (2008)
Paper in Proceedings of the 22nd International Conference on Computational Linguistics, pp. 865–872
Publisher: Association for Computational Linguistics - Native Judgments of Non-Native Usage: Experiments in Preposition Error Detection
Authors: J. Tetreault & M. Chodorow (2008)
Paper in COLING 2008: Proceedings of the Workshop on Human Judgments in Computational Linguistics, pp. 24–32
Publisher: Association for Computational Linguistics - Automated Scoring of Spontaneous Speech Using SpeechRater v1.0
Authors: X. Xi, D. Higgins, K. Zechner, & D. Williamson (2008)
ETS Research Report No. RR-08-62 - Towards Automatic Scoring of a Test of Spoken Language with Heterogeneous Task Types
Authors: K. Zechner & X. Xi (2008)
Paper in Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, pp. 98–106
Publisher: Association for Computational Linguistics
- Opportunities for Natural Language Processing in Education
- 2005–07
-
- On-the-Fly Customization of Automated Essay Scoring
Author: Y. Attali (2007)
ETS Research Report No. RR-07-42 - Toward More Substantively Meaningful Automated Essay Scoring
Authors: A. Ben-Simon & R. E. Bennett (2007)
Article in Journal of Technology, Learning, and Assessment, Vol. 6, No. 1 - Developing a Text Support Tool for English-Language Learners
Authors: J. Burstein, J. Shore, J. Sabatini, Y. Lee, & M. Ventura (2007)
Paper in Proceedings of the 13th International Conference on Artificial Intelligence in Education - The Automated Text Adaptation Tool
Authors: J. Burstein, J. Shore, J. Sabatini, Y. Lee, & M. Ventura (2007)
Paper in Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pp. 3–4
Publisher: Association for Computational Linguistics - Detection of Grammatical Errors Involving Prepositions
Authors: M. Chodorow, J. Tetreault, & N.-R. Han (2007)
Paper in Proceedings of the Fourth ACL-SIGSEM Workshop on Prepositions, pp. 25–30
Publisher: Association for Computational Linguistics - Reading Level Assessment for Literary and Expository Texts
Authors: K. M. Sheehan, I. Kostin, & Y. Futagi (2007)
Paper in Proceedings of the 29th Annual Meeting of the Cognitive Science Society
Publisher: The Cognitive Science Society - Document Similarity Measures to Distinguish Native vs. Non-Native Essay Writers
Authors: O. Gurevich & P. Deane (2007)
Paper in Proceedings of NAACL HLT 2007, Companion Volume (pp. 49–52)
Publisher: Association for Computational Linguistics - Electronic Essay Grading
Author: S.J. Haberman (2007)
Chapter in Handbook of Statistics: Psychometrics, Vol.26, pp. 205–233
Publisher: Elsevier - Item Distiller: Text Retrieval for Computer-Assisted Test Item Creation
Author: D. Higgins (2007)
ETS Research Memorandum No. RM-07-05 - Reliability of Human Annotation of Semantic Roles in Noisy Text
Author: D. Higgins (2007)
Paper in Proceedings of the International Conference on Semantic Computing (pp. 501–508)
Publisher: IEEE Computer Society - Sentence Similarity Measures for Essay Coherence
Authors: D. Higgins & J. Burstein (2007)
Paper in Proceedings of the Seventh International Workshop on Computational Semantics (IWCS-7) (pp. 77–88)
Publisher: International Workshop on Computational Semantics - Supporting Efficient, Evidence-Centered Item Development for the GRE Verbal Measure
Authors: K. M. Sheehan, I. Kostin, & Y. Futagi (2007)
ETS Research Report No. RR-07-29; GRE Board Research Report No. GREB-03-14 - SourceFinder: A Construct-Driven Approach for Locating Appropriately Targeted Reading Comprehension Source Texts
K. M. Sheehan, I. Kostin, & Y. Futagi (2007)
Paper in Proceedings of the 2007 Workshop of the International Speech Communication Association, Special Interest Group on Speech and Language Technology in Education - SpeechRater: A Construct-Driven Approach to Score Spontaneous Non-Native Speech
Authors: K. Zechner, D. Higgins, & X. Xi (2007)
Paper in Proceedings of the 2007 Workshop of the International Speech Communication Association (ISCA) Special Interest Group on Speech and Language Technology in Education (SLaTE)
Publisher: International Speech Communication Association - Toward an Understanding of the Role of Speech Recognition in Non-Native Speech Assessment
Authors: K. Zechner, I. Bejar, & R. Hemat (2007)
ETS Research Report No. RR-07-02 - Automated Essay Scoring with e-rater v.2.0
Authors: Y. Attali & J. Burstein (2006)
Article in Journal of Technology, Learning, and Assessment, Vol. 4, No. 3 - Moving the Field Forward: Some Thoughts on Validity and Automated Scoring
Author: R. E. Bennett (2006)
Chapter in Automated Scoring of Complex Tasks in Computer-Based Testing
Editors: D. M. Williamson, R. J. Mislevy, & I. I. Bejar
Publisher: Lawrence Erlbaum Associates - Rule-Based Methods for Automated Scoring: Application in a Licensing Context
Authors: H. Braun, I. I. Bejar & D.M. Williamson (2006)
Chapter in Automated Scoring of Complex Tasks in Computer-Scoring, pp. 83–123
Editors: D. M. Williamson, I. I. Bejar, R. J. Mislevy
Publisher: Lawrence Erlbaum Associates - Model Analysis and Model Creation: Capturing the Task-Model Structure of Quantitative Item Domains
Authors: P. Deane, E. A. Graf, D. Higgins, Y. Futagi, & R. Lawless (2006)
ETS Research Report No. RR-06-11 - Using Singular-Value Decomposition on Local Word Contexts to Derive a Measure of Constructional Similarity
Authors: P. Deane & D. Higgins (2006)
Chapter in Corpus Linguistics Beyond the Word — Corpus Research from Phrase to Discourse (pp. 43–58)
Publisher: Rodopi - Differences in Text Structure and its Implications for Assessment of Struggling Readers
Authors: P. Deane, K. Sheehan, J. Sabatini, & Y. Futagi (2006)
Article in Scientific Studies of Reading, Vol. 10, No. 3, pp. 257–276 - Detecting Errors in English Article Usage by Nonnative Speakers
Authors: N.-R. Han, M. Chodorow, & C. Leacock (2006)
Article in Natural Language Engineering, Vol. 12, No. 2, pp. 115–129 - Identifying Off-Topic Student Essays Without Topic-Specific Training Data
Authors: D. Higgins, J. Burstein, & Y. Attali (2006)
Article in Natural Language Engineering, Vol. 12, No. 2, pp. 145–159 - Inside SourceFinder: Predicting the Acceptability Status of Candidate Reading-Comprehension Source Documents
Authors: K. M. Sheehan, Y. Futagi, I. W. Kostin, R. Hemat, & D. I. Zuckerman (2006)
ETS Research Report No. RR-06-24 - Applications of Computers in Assessment and Analysis of Writing
Authors: M. D. Shermis, J. Burstein, & C. Leacock (2006)
Chapter in Handbook of Writing Research
Editors: C. A. MacArthur, S. Graham, & J. Fitzgerald
Publisher: Guilford Publications - Towards Automatic Scoring of Non-Native Spontaneous Speech
Authors: K. Zechner & I. Bejar (2006)
Paper in Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pp. 216–223
Publisher: Association for Computational Linguistics - Strategies for Evidence Identification through Linguistic Assessment of Textual Responses
Author: P. Deane (2006)
Chapter in Automated Scoring of Complex Tasks in Computer-Based Testing (pp. 313–372)
Editors: D. M. Williamson, R. J. Mislevy, & I. I. Bejar
Publisher: Lawrence Erlbaum Associates - Human Scoring
Authors: D. M. Williamson, R.J. Mislevy & I.I. Bejar (2006)
Chapter in Automated Scoring of Complex Task in Computer-Based Testing, pp. 49–82
Editors: D. M. Williamson, R. J. Mislevy, I.I. Bejar
Publisher: Lawrence Erlbaum Associates - Automated Scoring of Complex Tasks in Computer-Based Testing
Editors: D. M. Williamson, R.J. Mislevy & I. I. Bejar (2006)
Book
Publisher: Lawrence Erlbaum Associates - Automated Essay Scoring with e-rater v.2.0
Authors: Y. Attali & J. Burstein (2005)
ETS Research Report No. RR-04-45 - Advanced Capabilities for Evaluating Student Writing: Detecting Off-Topic Essays Without Topic-Specific Training
Authors: J. Burstein & D. Higgins (2005)
Paper in Proceedings of the International Conference on Artificial Intelligence in Education
Publisher: IOS Press - Translation Exercise Assistant: Automated Generation of Translation Exercises for Native-Arabic Speakers Learning English
Authors: J. Burstein & D. Marcu (2005)
Paper in Proceedings of HLT/EMNLP 2005 Interactive Demonstrations, pp. 16–17
Publisher: Association for Computational Linguistics - A Nonparametric Method for Extraction of Candidate Phrasal Terms
Author: P. Deane (2005)
Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pp. 605–613
Publisher: Association for Computational Linguistics - Multilingual Generalization of the ModelCreator Software for Math Item Generation
Authors: D. Higgins, Y. Futagi, & P. Deane (2005)
ETS Research Report No. RR-05-02 - Online Assessment in Writing
Authors: N. Horkay, R. E. Bennett, N. Allen, & B. Kaplan (2005)
Paper in Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-based Assessment Project (NCES 2005–457)
Editors: B. Sandene, N. Horkay, R. E. Bennett, N. Allen, J. Braswell, B. Kaplan, & A. Oranje
Publisher: U.S. Department of Education, National Center for Education Statistics - Online Assessment in Mathematics
Authors: B. Sandene, R. E. Bennett, J. Braswell, & A. Oranje (2005)
Paper in Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-based Assessment Project (NCES 2005–457)
Editors: B. Sandene, N. Horkay, R. E. Bennett, N. Allen, J. Braswell, B. Kaplan, & A. Oranje
Publisher: U.S. Department of Education, National Center for Education Statistics
- On-the-Fly Customization of Automated Essay Scoring
- 2001–04
-
- Automated Essay Evaluation: The Criterion Online Writing Evaluation Service
Authors: J. Burstein, M. Chodorow, & C. Leacock (2004)
Article in AI Magazine, Vol. 25, No. 3, pp. 27–36 - Beyond Essay Length: Evaluating e-rater's Performance on TOEFL® Essays
Authors: M. Chodorow & J. Burstein (2004)
TOEFL® Research Rep. No. RR-73, ETS Research Report No. RR-04-04 - Statistical and Measurement Properties of Features Used in Essay Assessment
Author: S.J. Haberman (2004)
ETS Research Report No. RR-04-21 - Detecting Errors in English Article Usage with a Maximum Entropy Classifier Trained on a Large, Diverse Corpus
Authors: N.-R. Han, M. Chodorow, & C. Leacock (2004)
Paper in Proceedings of the 4th International Conference on Language Resources and Evaluation
Publisher: European Language Resources Association - Evaluating Multiple Aspects of Coherence in Student Essays
Authors: D. Higgins, J. Burstein, D. Marcu, & C. Gentile (2004)
Paper in HLT-NAACL 2004: Main Proceedings, pp. 185–192
Editors: S. Dumais, D. Marcu, & S. Roukos
Publisher: Association for Computational Linguistics - Keynote Address: Statistical Analysis of Text in Educational Measurement
Author: C. Leacock (2004)
Paper in Proceedings of the 7th International Conference on Textual Data Statistical Analysis
Editors: G. Purnelle, C. Fairon, & A. Dister
Publisher: Presses Universitaires de Louvain - Automated Tools for Subject Matter Expert Evaluation of Automated Scoring
Authors: D. Williamson, I.I. Bejar & A. Sax (2004)
Article in Applied Measurement in Education, Vol.17, No. 4, pp. 323–357 - Criterion: Online Essay Evaluation: An Application for Automated Evaluation of Student Essays
Authors: J. Burstein, M. Chodorow, & C. Leacock (2003)
Paper in Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence
Publisher: Association for the Advancement of Artificial Intelligence - Automated Evaluation of Discourse Structure in Student Essays
Authors: J. Burstein & D. Marcu (2003)
Chapter in Automated Essay Scoring: A Cross-disciplinary Perspective, pp. 200–219
Editors: M. D. Shermis & J. Burstein
Publisher: Lawrence Erlbaum Associates, Inc. - A Machine Learning Approach for Identification of Thesis and Conclusion Statements in Student Essays
Authors: J. Burstein & D. Marcu (2003)
Article in Computers and the Humanities, Vol. 37, No. 4, pp. 455–467 - Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays
Authors: J. Burstein, D. Marcu, & K. Knight (2003)
Article in IEEE Intelligent Systems: Special Issue on Advances in Natural Language Processing, Vol. 18, No. 1, pp. 32–39 - Toward Evaluation of Writing Style: Finding Overly Repetitive Word Use in Student Essays
Authors: J. Burstein & M. Wolska (2003)
Paper in Proceedings of the 10th Conference of the European chapter of the Association for Computational Linguistics
Publisher: Association for Computational Linguistics - The e-rater Scoring Engine: Automated Essay Scoring with Natural Language Processing
Author: J. Burstein (2003)
Chapter in Automated Essay Scoring: A Cross-Disciplinary Perspective, pp. 107–115
Editors: M. D. Shermis & J. Burstein
Publisher: Lawrence Erlbaum Associates, Inc. - Co-occurrence and Constructions
Author: P. Deane (2003)
Chapter in Determination of information and tenor in texts: Multidisciplinary approaches to discourse 2003
Editors: L. Lagerweft, W. Spooren, & L. Degand
Publisher: Stichting Neerlandistiek & Nodus Publikationen - Improving the Statistical Aspects of E-rater: Exploring Alternative Feature Reduction & Combination Rules
Authors: X. Feng, N. J. Dorans, L. N. Patsula & B. Kaplan (2003)
ETS Research Report No. RR-03-15 - Automated Grammatical Error Detection
Authors: C. Leacock & M. Chodorow (2003)
Chapter in Automated Essay Scoring: A Cross-Disciplinary Perspective, pp. 186–199
Editors: M. D. Shermis & J. Burstein
Publisher: Lawrence Erlbaum Associates, Inc. - Directions in Automated Essay Scoring
Authors: J. Burstein & M. Chodorow (2003)
Chapter in Handbook of Applied Linguistics, pp. 487–497
Editor: R. Kaplan
Publisher: Oxford University Press - Technology and Assessment
Authors: L. Frase, R. Almond, J. Burstein, K. Kukich, R. Mislevy, K. Sheehan, L. Steinberg, K. Singley, & M. Chodorow (2003)
Chapter in Technology Applications in Education: A Learning View
Editors: R. S. Perez & H. F. O'Neill Jr.
Publisher: Lawrence Erlbaum Associates - c-rater: Scoring of Short-Answer Questions
Authors: C. Leacock & M. Chodorow (2003)
Computers and the Humanities, Vol. 37, pp. 389–405 - Automated Essay Scoring: A Cross-Disciplinary Perspective
Authors: M. D. Shermis & J. Burstein (2003)
Publisher: Lawrence Erlbaum Associates - Techniques for Detecting Syntactic Errors in Text
Authors: M. Chodorow & C. Leacock (2002)
Paper in Technical Report of the Institute of Electronics, Information, and Communication Engineers, Vol. 102, No. 491, pp. 37–41 - Modeling the Difficulty of Quantitative Reasoning Items: Implications for Item Generation
Authors: M. K. Enright & K. M. Sheehan (2002)
Chapter in Item Generation for Test Development, pp. 129-157
Editors: S. H. Irvine & P. C. Kyllonen
Publisher: Lawrence Erlbaum Associates - Electronic Sources as Input to GRE Reading Comprehension Item Development: SourceFinder Prototype Evaluation
Authors: R. Passonneau, L. Hemat, J. Plante, & K. M. Sheehan
ETS Research Report No. RR-02-12 - Towards Automatic Classification of Discourse Elements in Essays
Authors: J. Burstein, D. Marcu, S. Andreyev, & M. Chodorow (2001)
Paper in Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics, pp. 98–105
Publisher: Association for Computational Linguistics - Automatic Assessment of Vocabulary Usage Without Negative Evidence
Authors: C. Leacock & M. Chodorow (2001)
TOEFL® Research Rep. No. 67, ETS RR-01-21 - Stumping e-rater: Challenging the Validity of Automated Essay Scoring
Authors: D. E. Powers, J. Burstein, M. Chodorow, M. E. Fowles, & K. Kukich (2001)
GRE® Board Professional Rep. No. 98-08bP, ETS Research Report No. RR-01-03 - An Inquiry into the Nature of the Sentence-Completion Task: Implications for Item Generation
Authors: K. M. Sheehan & R. J. Mislevy (2001)
ETS Research Report No. RR-01-13
- Automated Essay Evaluation: The Criterion Online Writing Evaluation Service
- pre-2001
-
- Comparing the Validity of Automated and Human Essay Scoring
Authors: D. E. Powers, J. Burstein, M. Chodorow, M. E. Fowles, & K. Kukich (2000)
GRE Board Research Rep. No. 98-08a, ETS Research Report No. RR-00-10 - An Unsupervised Method for Detecting Grammatical Errors
Authors: M. Chodorow & C. Leacock (2000)
Paper in Proceedings of the 1st Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pp. 140–147
Publisher: Association for Computational Linguistics - Benefits of Modularity in an Automated Scoring System
Authors: J. Burstein & D. Marcu (2000)
Proceedings of the Workshop on Using Toolsets and Architectures to Build NLP Systems, 18th International Conference on Computational Linguistics, Luxembourg
Publisher: Association for Computational Linguistics - Automated Essay Scoring for Nonnative English Speakers
Authors: J. Burstein & M. Chodorow (1999)
Proceedings of the ACL99 Workshop on Computer-Mediated Language Assessment and Evaluation of Natural Language Processing
Editor: M. B. Olson
Publisher: Association for Computational Linguistics - 'Mental Model' Comparison of Automated and Human Scoring
Authors: D.M. Williamson, I.I. Bejar & A.S. Hone (1999)
Article in Journal of Educational Measurement, Vol. 36, No. 2, pp. 158–184 - Validity and Automated Scoring: It's Not Only the Scoring
Authors: R. E. Bennett & I. I. Bejar (1998)
Article in Educational Measurement: Issues and Practice, Vol. 17, No. 4, pp. 9–17 - Scoring Essays Automatically Using Surface Features
Authors: R. M. Kaplan, S. E. Wolff, J. C. Burstein, C. Lu, D. A. Rock, & B. A. Kaplan (1998)
GRE Board Professional Rep. No 94-21P; ETS Research Report No. RR-98-39 - Enriching Automated Scoring Using Discourse Marking
Authors: J. Burstein, K. Kukich, S. Wolff, C. Lu, & M. Chodorow (1998)
Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics
Publisher: Association for Computational Linguistics - Automated Scoring Using a Hybrid Feature Identification Technique
Authors: J. Burstein, K. Kukich, S. Wolff, C. Lu, M. Chodorow, L. Braden-Harder, & M. D. Harris (1998)
Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Vol. 1, pp. 206–210
Publisher: Association for Computational Linguistics - 'Mental Model' Comparison of Automated & Human Holistic Scoring of Architectural Design Simulations
Authors: D. M. Williamson, I.I. Bejar & A.S. Hone (1998)
Article in Clear Exam Review, Vol. 9, No. 1, pp. 28–35 - Evaluating an Automatically Scorable, Open-Ended Response Type for Measuring Mathematical Reasoning in Computer-Adaptive Tests
Authors: R. E. Bennett, M. Steffen, M. K. Singley, M. Morley, & D. Jacquemin (1997)
Article in Journal of Educational Measurement, Vol. 34, pp. 163–177 - Automatic Scoring of Advanced Placement Biology Essays
Authors: J. Burstein, R. Kaplan, S. Wolff, & C. Lu (1997)
ETS Research Report No. RR-97-22 - Validation and Extension of the Mathematical Expression Response Type: Applications of Schema Theory to Automatic Scoring and Item Generation in Mathematics
Authors: M. K. Singley & R. E. Bennett (1997)
ETS Research Report No. RR-97-19 - Using Lexical Semantic Techniques to Classify Free-Responses
Authors: J. Burstein, R. Kaplan, S. Wolff, & C. Lu (1996)
Breadth and Depth of Semantic Lexicons: Proceedings of a Workshop Sponsored by the Special Interest Group on the Lexicon of the Association for Computational Linguistics, pp. 20–29
Publisher: Association for Computational Linguistics - Evaluating a Prototype Essay Scoring Procedure Using Off-the-Shelf Software
Authors: R. M. Kaplan, J. Burstein, H. Trenholm, & C. Lu (1995)
ETS Research Report No. RR-95-21 - GE FRST Evaluation Report: How Well Does a Statistically-Based Natural Language Processing System Score Natural Language Constructed Responses?
Authors: J. Burstein & R. M. Kaplan (1995)
ETS Research Report No. RR-95-29 - On the Application of Context to Natural Language Processing Applied to the Analysis of Test Responses
Authors: J. C. Burstein & R. M. Kaplan (1995)
Paper in Proceedings of the IJCAI-95 Workshop on Context in Natural Language Processing
Publisher: International Joint Conferences on Artificial Intelligence - Using the Free-Response Scoring Tool to Automatically Score the Formulating-Hypotheses Item
Authors: R. M. Kaplan & R. E. Bennett (1994)
ETS Research Report No. RR-94-08 - Environments for Presenting and Automatically Scoring Complex Constructed-Response Items
Author: R. E. Bennett (1994)
Chapter in Personnel Selection and Classification
Editors: M. G. Rumsey, C. B. Walker, & J. H. Harris
Publisher: Erlbaum - Constructed Response & Differential Item Functioning: A Pragmatic Approach
Authors: N. J. Dorans, A. P. Schmitt (1993)
Chapter in Construction Versus Choice in Cognitive Measurement, pp. 135–165
Editors: R.E. Bennett & W. C. Ward
Publisher: Routledge - A Review of Automatically Scorable Constructed-Response Item Types for Large-Scale Assessment
Authors: M. E. Martinez & R. E. Bennett (1992)
Article in Applied Measurement in Education, Vol. 5, pp. 151–169 - Agreement Between Expert System and Human Raters' Scores on Complex Constructed-Response Quantitative Items
Authors: M. M. Sebrechts, R. E. Bennett, & D. A. Rock (1991)
Article in Journal of Applied Psychology, Vol. 76, pp. 856–862 - Expert-System Scores for Complex Constructed-Response Quantitative Items: A Study of Convergent Validity
Authors: R. E. Bennett, M. M. Sebrechts, & D. A. Rock (1991)
Article in Applied Psychological Measurement, Vol. 15, pp. 227–239 - Using a Trainable Pattern-Directed Computer Program to Score Natural Language Item Responses
Author: R. M. Kaplan (1991)
ETS Research Report No. RR-91-31 - Assessment of an Expert System's Ability to Grade and Diagnose Automatically Students' Constructed Responses to Computer Science Problems
Authors: R. E. Bennett et. al. (1990)
Chapter in Artificial Intelligence and the Future of Testing, pp. 293–320
Publisher: Erlbaum - The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items
Authors: R. E. Bennett et. al. (1990)
Article in Applied Psychological Measurement, Vol. 14, pp. 151–162 - Scoring Constructed Responses Using Expert Systems
Authors: H. I. Braun, R. E. Bennett, D. Frye, & E. Soloway (1990)
Article in Journal of Educational Measurement, Vol. 27, pp. 93–108
- Comparing the Validity of Automated and Human Essay Scoring