By: Hans Sandberg
It is more important than ever to be able to read fluently and understand what one reads. However, one-third of fourth-graders in the United States don't reach even the basic level of reading, according to the National Assessment of Educational Progress (NAEP). Focus on ETS R&D talked to four scientists about their work on an interactive book reading app. Relay Reader™ is built to support the development of the learner's reading fluency and stamina, while enjoying a good story.
Why are you building a digital reading app?
Beata Beigman Klebanov: Reading is a tool that belongs in everyone's toolbox. To read for learning and pleasure, one must be a fluent reader who understands what one reads — but that is not the case today. We hope that our Relay Reader app can help change that by delivering an immersive, engaging and extended reading experience. Our goal with the app is to give children and adults, who are developing their English-language literacy, a chance to improve their skills, while also enjoying a good story without stress or frustration. They can do this by taking turns reading aloud with a skilled and engaging audiobook narrator. The app records students' reading and can measure their reading fluency unobtrusively, without interrupting the flow of reading.
Why is it important for students to read text aloud?
John Sabatini (University of Memphis): There are many situations in school and later in adult life where you need to be able to read something aloud. Maybe a teacher asks a student to read a text aloud, or an employee or customer needs to reiterate something in a meeting or over the phone. To not be able to read aloud can be embarrassing and interfere with learning, job performance and social interactions. There is a growing body of evidence showing that the very skills used in oral reading overlap with the processes used to understand something when reading silently.
Why not simply give students a popular book to read?
Beata Beigman Klebanov: It can be quite daunting for developing readers to tackle an entire book on their own. We help them by having the app read about half of the text and then quizzing them on the parts that they have just read. We can also track their reading behavior and reading competency and share this information with their teachers, who can then use the feedback from the app to encourage or help the child when necessary. So far, the reaction from both children and teachers has been very enthusiastic.
How do you measure reading fluency?
Anastassia Loukina: The technology we use to measure reading fluency is also used by ETS's automated speech scoring engines. It transcribes the recordings and compares the result to the text in the book. We know from experience that the audio quality is very important for an accurate result, which should not come as a surprise to anybody who has ever used a virtual assistant or navigated a voice menu. These systems work better if you speak clearly in a quiet place.
When we use automated speech scoring in assessments, we have different methods for making sure that the scoring engine can work on a clear recording. We try to control the conditions of the assessment centers and sometimes we ask the learners to rerecord their responses — but these solutions can be distracting for a student using the Relay Reader app. We want them to enjoy reading with minimal interruptions, and we want them to be able to read in places where they are comfortable. We also don't want to have them reread their passages because of background noise, or because the reader stopped in the middle of the page to chat with a friend.
But the good news is that we get a lot of data from each learner. A reader who used the app to finish a 300-page book will have taken more than 200 turns and answered more than 200 comprehension questions. Our analysis of several pilot studies consistently shows that this rich data is likely to give us enough information to obtain estimates of a child's reading skills, even if the recordings for some of these turns are not as clean as we would like them to be. Our research for this project focuses on how to make the best of the data, and how to help learners and provide additional support.
What challenges have you faced in building the app and what do you plan to do next?
Nitin Madnani: Reading a good book is something you'd rather do while curled up on comfy cushions, than sitting at a desk or in a computer lab. That's why we designed Relay Reader to be used on tablets, smartphones, PCs and laptops. The reader can seamlessly switch between different devices without losing their bookmarks, so they can easily continue reading where they stopped.
Another challenge was that mobile technologies develop so fast that state-of-the art apps might not work well on older or less powerful devices. We have worked with the app developers to make sure that our app runs on devices you might find in schools and summer camps. We have pilot-tested the app with third- to fifth-graders in summer camps, and we are currently using it in our first full-length trial in one school. We are also piloting the mobile app in other contexts — for example, with low-literacy adult learners. Finally, while the web version of the app is already available publicly, we are also planning to release a public version of the mobile app that will be available to anybody who wants to try it. We hope that it encourages learners to read more and have fun doing it, which we hope will make them better readers.
Find out more about how Relay Reader can improve the reading skills of children.
Beata Beigman Klebanov, Anastassia Loukina and Nitin Madnani are Research Scientists in the Natural Language Processing and Speech Group in the ETS Research & Development division. John Sabatini is a professor at the University of Memphis. He was previously a Managing Principal Research Scientist at ETS.
Beigman Klebanov, B., Loukina, A., Lockwood, J., Liceralde, V., Sabatini, J., Madnani, N., Gyawali, B., Wang, Z., & Lentini, J. (2020). Detecting learning in noisy data: The case of oral reading fluency. In Proceedings of the 10th International Conference on Learning Analytics Knowledge (LAK20). ACM, New York, NY, USA, pp. 490–495. https://dl.acm.org/doi/abs/10.1145/3375462.3375490
Loukina, A., Beigman Klebanov, B., Lange, P., Qian, Y., Gyawali, B., Madnani, N., Misra, A., Zechner, K., Wang, Z., & Sabatini, J. (2019) Automated Estimation of Oral Reading Fluency During Summer Camp e-Book Reading with MyTurnToRead. Proc. Interspeech 2019, 21–25, DOI: 10.21437/Interspeech.2019-2889.
Madnani, N., Beigman Klebanov, B., Loukina, A., Gyawali, B., Sabatini, J., Lange, P., & Flor, M. (2019). My Turn To Read: An Interleaved E-book Reading Tool for Developing and Struggling Readers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Florence, Italy, 2019, pp. 141–146. https://www.aclweb.org/anthology/P19-3024.
"NAEP Report Cards." The Nation's Report Card, The National Assessment Governing Board (NAGB), 2017. www.nationsreportcard.gov/.
Sabatini, J., Wang, Z., & O'Reilly, T. (2019). Relating Reading Comprehension to Oral Reading Performance in the NAEP Fourth‐Grade Special Study of Oral Reading. Reading Research Quarterly, 54(2), 253–271. https://doi.org/10.1002/rrq.226