Reading for Understanding — Publications

These are some recent publications that ETS and our collaborators have authored related to the assessment portion of the Reading for Understanding (RfU) initiative.

2016

• Assessing Comprehension in Kindergarten Through Third Grade
  J. Sabatini, L. Halderman, T. O’Reilly, & J. Weeks
  Topics in Language Disorders, 36, 4, 334–355.

  The authors explore whether it is feasible to develop a more integrated, scenario-based assessment of comprehension for younger students. They also studied developmental differences in item performance when administration was in listening versus reading modalities. Read the publisher’s full abstract.

• Using Advances in Cognitive Science to Improve Students Study Skills and Reading Comprehension
  T. O’Reilly, & J. Sabatini
  In R. Schiff and R. M. Joshi (eds), Interventions in Learning Disabilities: Literacy Studies, 13, (pp. 139–158). Switzerland: Springer International.

  The authors of this chapter review recent developments from the cognitive literature that can potentially shed light on solutions for improving students study skills, as well as their general reading comprehension. Read the publisher’s full abstract.

2015

• Improving Comprehension Assessment for Middle and High School Students: Challenges and Opportunities
  J. Sabatini, Y. Petscher, T. O’Reilly, & A. Truckenmiller
  In D. Reed and K. Santi (eds). Improving reading comprehension of middle and high school students, (pp. 119–151). New York: Springer.

  The authors discuss standardized reading assessments, focusing especially on assessment designs and analytics in order to see how they can reflect a balance of practical and measurement constraints. They also cover advances in the learning sciences, measurement, and electronic technologies, and their impact on the design of future reading assessments. Read the publisher’s full abstract.

• SARA Reading Components Tests, RISE Forms: Technical Adequacy and Test Design, 2nd Edition
  J. Sabatini, K. Bruce, Steinberg, & J. Weeks
  ETS Research Report No. RR-15–32

  The authors of this technical report describe the conceptual foundation and measurement properties of the Reading Inventory and Scholastic Evaluation (RISE) and review the theoretical and empirical foundations of each subtest and the item designs. View citation record.

2014

• Broadening the Scope of Reading Comprehension Using Scenario-based Assessments: Preliminary Findings and Challenges
  J. Sabatini, T. O’Reilly, L. Halderman, & K. Bruce
  International Journal Topics in Cognitive Psychology, 114, 693–723

  This paper presents the results of a study that evaluated a scenario-based assessment of reading comprehension of sixth-grade students. The assessment was designed to measure the students' ability to integrate and evaluate a set of thematically related sources. View the abstract or download the full paper.

• Measurement: Facilitating the Goal of Literacy
  J. Gorin, T. O’Reilly, J. Sabatini, Y. Song, & P. Deane
  In B. Miller, P. McCardle, & R. Long (eds), Teaching reading and writing: Improving instruction and student achievement, (pp. 119–128). Baltimore, MD: Paul H. Brookes Publishing Co.

  The authors discuss four key areas supporting innovations in assessment for literacy instruction that focus on reading, writing, and their connection. In particular, they describe how advances in (a) cognitive models, (b) task design, (c) automated scoring and (d) psychometric modeling can work in concert to create a more effective assessment system. View citation record.

• Designing Reading Comprehension Assessments for Reading Interventions: How a Theoretically Motivated Assessment Can Serve as an Outcome Measure
  T. O’Reilly, J. Weeks, J. Sabatini, L. Halderman, & J. Steinberg
  Educational Psychology Review, Vol. 26, No. 3, pp. 403–424

  This paper describes and evaluates three summative reading comprehension assessments designed in conjunction with an intervention team. The authors discuss the empirical work that motivated the assessments, the aims of the intervention, the process they used to develop new assessments, and the possible implications for intervention and assessment. Read the publisher’s full abstract.

• Integrating Scenario-based and Component Reading Skill Measures to Understand the Reading Behavior of Struggling Readers
  J. Sabatini, T. O’Reilly, L. Halderman, & K. Bruce
  Learning Disabilities Research & Practice, Vol. 29, No.1, pp. 36–43

  This article presents data from two assessments that were designed to provide a more complete picture on the performance of struggling readers. In particular, the authors present the results from a scenario-based measure of reading comprehension and measure component reading skills. View citation record.

2013

• Reading for Understanding: How Performance Moderators and Scenarios Impact Assessment Design
  T. O’Reilly & J. Sabatini
  ETS Research Report No. RR-13-31

  This paper represents the third installment of the Reading for Understanding assessment framework. It builds upon two prior installments by examining the role of performance moderators and the use of a scenario-based assessment technique in reading test design. View citation record.

• Preliminary Reading Literacy Assessment Framework: Foundation and Rationale for Assessment and System Design
  J. Sabatini, T. O’Reilly, & P. Deane
  ETS Research Report No. RR-13-30

  This paper represents the second installment of the Reading for Understanding assessment framework. This paper describes the foundation and rationale of the assessment, the target constructs to be assessed, and general principles underlying the assessment design. View citation record.

• Automated Scoring of a Summary Writing Task Designed to Measure Reading Comprehension
  N. Madnani, J. Burstein, J. Sabatini, & T. O’Reilly
  Paper in Proceedings of the Eighth Workshop on Innovative Use of Natural Language Processing for Building Educational Applications, pp. 163–168

  This paper provides an initial evaluation of automated approaches for scoring constructed responses in the context of the Reading for Understanding project. The authors presented results showing that the automated approach performed well on student-written summaries for two different passages. View citation record.

• Rationale for a New Generation of Reading Comprehension Assessments
  J. Sabatini & T. O'Reilly
  B. Miller, L. Cutting, & P. McCardle (Eds)
  Unraveling Reading Comprehension: Behavioral, Neurobiological, and Genetic Components pp. 100–111. Publisher: Brookes Publishing

  This paper presents the first part of the RfU assessment framework. The paper provided the rationale for a new generation of reading assessments and described six principles for assessment design. Read the publisher's full abstract.

2012

• Middle School Reading Assessment: Measuring What Matters Under a RTI framework. Reading Psychology Special Issue: Response to Intervention
  T. O'Reilly, J. Sabatini, J. K. Bruce, S. Pillarisetti, & C. McCormick
  Reading Psychology, Vol. 33, No. 1–2, pp. 162–189

  In this article, the authors present a prototype reading assessment of component and integrated reading skills. The work is part of an effort to conceptualize, prototype and test aspects of a framework in the context of the Response to Intervention approach. Read the publisher's full abstract.

• Psychometric Analysis of the Diagnostic Evaluation of Language Variation Assessment
  Y. Petscher, C. M. Conner, & S. Al Otaiba
  Assessment for Effective Intervention, Vol. 37, No. 4, pp. 243–250

  This article describes a study of the psychometrics of the Diagnostic Evaluation of Language Variation-Screening Test (DELV-S). Read the publisher's full abstract.

• Measuring Up: Advances in How We Assess Reading Ability
  Editors: J. Sabatini, E. R. Albro, & T. O'Reilly
  Publisher: Rowman & Littlefield

  In this book, educational researchers question the traditional format of reading comprehension tests and offer ideas and innovations for future reading assessments. Read the publisher's full description of this book.

• Reaching an Understanding: Innovations in How We View Reading Assessment
  Editors: J. Sabatini, T. O'Reilly & E. R. Albro
  Publisher: Rowman & Littlefield

  This follows the editors' arguments in the previous volume, Measuring Up: Advances in How We Assess Reading Ability, by describing some early attempts to apply theory to new assessments and measurement models. Read the publisher's full description of this book.