angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

Automated Analysis of Text in Graduate School Recommendations

Author(s):
Heilman, Michael; Breyer, F. Jay; Williams, Frank; Klieger, David M.; Flor, Michael
Publication Year:
2015
Report Number:
RR-15-23
Source:
ETS Research Report
Document Type:
Report
Page Count:
14
Subject/Key Words:
Letters of Recommendation Automated Scoring and Natural Language Processing

Abstract

Graduate school recommendations are an important part of admissions in higher education, and natural language processing may be able to provide objective and consistent analyses of recommendation texts to complement readings by faculty and admissions staff. However, these sorts of high-stakes, personal recommendations are different from the product and service reviews studied in much of the research on sentiment analysis. In this report, we develop an approach for analyzing recommendations and evaluate the approach on four tasks: (a) identifying which sentences are actually about the student, (b) measuring specificity, (c) measuring sentiment, and (d) predicting recommender ratings. We find substantial agreement with human annotations and analyze the effects of different types of features.

Read More