By: Hans Sandberg
We tend to take reading for granted, but just about one in three students in grades four and eight were "Proficient" or better at reading, according to "The Nation’s Report Card." These alarming numbers underscore the need for a nationwide intervention to make K–12 students better readers. Focus on ETS R&D talked to John Sabatini and Tenaha O’Reilly about the role of ETS research in improving reading for understanding.
O’Reilly and Sabatini led a team of ETS researchers participating in the federal Reading for Understanding (RfU) initiative, which sought to improve reading comprehension through intervention and assessment. Concluding in 2016, the five-year program involved over 130 researchers in linguistics, cognitive psychology, developmental psychology, reading instruction, speech and language pathology, assessment and evaluation. The ETS team worked on innovative summative assessments for students in grades preK through 12, while five other teams (led by Florida State University [FSU], The Ohio State University, the Strategic Education Research Partnership, University of Illinois at Chicago and University of Texas at Austin) worked primarily on reading interventions. The project was designed to stimulate collaboration and cross-disciplinary thinking as well as the use of alternative methodologies and technologies for assessment design. It was funded by the Institute of Education Sciences (IES) at the U.S. Department of Education.
What is it that we don’t know about reading?
John Sabatini: Reading used to be about understanding the content of books, magazines, newspapers and letters, but today you need to add the entire universe of Internet documents and other communications on all kinds of media, from printed documents to texts on tablets and smart watches.
Tenaha O’Reilly: To be proficient in reading, people must be able to access multiple sources of text and related materials, often digital in format. To understand what they read, readers must integrate and evaluate what they read. Reading has become more instrumental, so that you read for a particular purpose. The act of reading is also more social, since you often read in a social setting, like a chat, an email or text exchange.
Why is it so important to be able to read well?
John Sabatini: The simple answer is that you need this skill to be successful in life and work. It will be very hard to get the training and education you need if you can’t read and understand written material, or communicate with digital media. It used to be possible to find good paying jobs with only a high school education, but that’s no longer the case. Employers ask for increasingly higher levels of education and skills, especially for well-paid jobs. Today, you can hardly get a job if you can’t fill out an online job application and submit a digital resume.
Tenaha O’Reilly: This is the new reality of the 21st century and it will present major challenges for school children who cannot read at grade level.
What did ETS researchers focus on in the RfU initiative?
John Sabatini: Reading is a complex activity that begins even before a young child enters school and grows in sophistication across time, hopefully at least until the individual reaches the proficiency necessary for college and career readiness. Unfortunately, many students go astray on this developmental path, and could use additional support. To address their needs, we worked with a collaborative team of researchers to develop a suite of assessments that covers different aspects of the reading construct across the student’s development. This suite approach allows for greater flexibility, as educators can pick and choose the assessments that best fit their needs.
The main goal of our team was to develop and evaluate the next generation of assessments of reading comprehension skills. We worked with researchers from Florida State University (FSU), Arizona State University (ASU) and Northern Illinois University (NIU) to do this.
The three teams brought a range of different skills to the table. FSU had a team with extensive experience with interim assessment from working with ETS, ASU’s team provided eye tracking experience to help uncover how people read in testing situations, and NIU’s team helped us better understand how people read and integrate multiple texts. ETS already has a strong history of building innovative reading assessments through our work in the Cognitively Based Assessment of, for, and as Learning (CBAL™) initiative, as well as our work with struggling students.
What did the ETS-led team achieve across the RfU project?
John Sabatini: We built a lot of reading assessment forms, collected data on their performance and tried to learn more about student skills. We created two main types of assessments that together produce a more complete picture of reading ability. The first, Reading Inventory and Scholastic Evaluation (RISE), identifies students’ strengths and weaknesses in foundational reading skills such as decoding. The second, Global, Integrated, Scenario-based Assessments (GISA), measures higher level comprehension skills in a digital learning environment. GISA incorporates a design feature called scenario-based assessment. Both have been administered to a wide range of students. We have collected data to begin the process of creating scale scores to link GISA forms or RISE forms across grades. This would make it possible to use the scores to monitor changes in reading over time, or to evaluate reading interventions. In short, we now have assessments for a range of purposes.
What key innovations came out of the RfU project?
Tenaha O’Reilly: One is scenario-based assessment, which is used across the CBAL initiative. This technique is used to deliver a set of inter-related reading materials from different sources in a digital environment. We may ask test takers questions that provide readers with a purpose, for example: "Should our neighborhood create a community garden?" Next, they may be asked to look at a mix of materials written by different authors and perspectives, for example, an email, blog or website. As a culminating task, they could be asked to prepare a flyer that informs community members of the pros and cons of creating a community garden. Throughout the course of the assessment, students are asked to integrate, evaluate and synthesize information to solve a key problem or make a decision.
How can better assessments lead to better reading?
Tenaha O’Reilly: Tests can by themselves foster a positive learning environment, and properly designed tests can potentially help students build understanding and model good habits of mind, and measure learning. Global, Integrated, Scenario-based Assessments (GISA) are designed with these features in mind, not only to potentially help students improve their understanding, but also to help teachers identify classroom-level strengths and weaknesses in various reading skills. There is nothing that says tests must be stressful, boring or be perceived as barriers to learning. By integrating the literature from the learning sciences, we can better understand how students learn and incorporate that into the assessment. This way, we can not only measure reading achievement, but promote the power of learning through assessment.
For instance, with GISA we present items in a sequence to see what part of a more complex construct students can or can’t handle. We provide guidelines to help students evaluate websites or summaries. We include simulated peers to help make the test less threatening, to provide hints or to elicit desired processing. Since the scenarios are based on topics, we measure background knowledge before the test taker reads passages. Often we present these same questions again later to determine if the students learned from the assessment. Because the assessments are based on a thematic topic, it gives us a chance to dig deeper into issues. When students are less familiar with a topic, the structure provides an opportunity for students to learn something about it.
How are you sharing the findings of your research?
John Sabatini: A major aim of this federally funded grant was to share innovations and insights so that they can be applied in other areas, which is consistent with ETS’s mission. We had a very good opportunity to share our findings and learn from others at a National Symposium on Reading for Understanding, which was held in Alexandria, Va., on May 18–19, 2016. The event was co-hosted by ETS and the Council of Chief State School Officers (CCSSO), and co-sponsored by 13 other organizations. It brought together about 160 state and district educational leaders, researchers, educational product companies and policymakers.
Tenaha O’Reilly: One of the benefits of being part of ETS R&D is that we have access to major research and assessment projects across the nation and the world. We were, for example, asked by the National Center for Education Statistics (NCES) to provide the reading battery for its Middle Grades Longitudinal Study (MGLS). This is the first national study to track middle school students’ development in cognitive, emotional and related success factors, and it will use an adapted version of GISA and RISE assessments to track reading development.
John Sabatini is a Principal Research Scientist in ETS’s R&D division. Tenaha O’Reilly is a Senior Research Scientist in ETS’s R&D division.