Education Experts Discuss Next Generation of Reading Assessments

Contact:

Princeton, N.J. (March 13, 2009) —

ETS researchers recently addressed a gathering of education experts in Washington, D.C., as part of the Educational Testing Service (ETS) Research Forum “Toward the Next Generation of Reading Assessment.” The seminar described experimental work that ETS has been conducting around new approaches to K–12 assessment.

The seminar featured presentations by Randy Elliot Bennett, Distinguished Research Scientist; Kathleen M. Sheehan, Principal Research Scientist; and Tenaha O’Reilly, Research Scientist, all in ETS’ Research & Development (R&D) Division. The forum was the fourth such gathering. 

“Given the important role education plays in ensuring any nation’s future, the critical question is, ‘Can we have a balanced system of assessment that provides the necessary policy information without undermining classroom instruction?’” explained Bennett. “This should be a system that documents what students have achieved, helps identify how instruction should be planned and adjusted, and is considered by students and teachers to be a worthwhile educational experience in and of itself. This is the vision we explored.”

Bennett further told attendees, “A key problem with today's school accountability assessments is that they are grounded in an outdated scientific model for conceptualizing proficiency, teaching it and measuring it. Because we have conceptualized and implemented accountability assessment as a one-time event, our measures are unavoidably shallow in what they assess — composed of many short items that test component skills in isolation. While those items may assess important learning, they inevitably omit critical aspects of what students must know and be able to do to be successful in a 21st-century world."

Attendees learned about ETS’s team of cognitive scientists, test developers, psychometricians and technology specialists who are developing prototype summative and formative assessments based on modern cognitive scientific research. These prototype assessments are being piloted and refined in collaboration with the Portland School District, arranged through the Maine Department of Education.

“Traditional standardized and formative assessments have been successful at achieving measurement goals and provide reliable evidence at a reasonable cost,” Sheehan told attendees. “However, they also push curriculum and instruction in the wrong direction by testing isolated skills, which leads to teaching isolated skills.”

What’s needed, she said, is “a new type of accountability assessment that specifically addresses the tradeoffs between measurement and learning goals by retaining some features of traditional standards-based assessments and some features of performance assessments.”

The team also presented a prototype reading assessment that included a set of scenario-based tasks designed to give students a realistic purpose for reading a collection of related stimulus materials. Individual tasks required students to work with multiple documents to achieve the specified purpose. “These tasks are designed to be more like reading in nontesting situations,” said Sheehan. The presentation also illustrated how individual tasks might help teachers identify where their students stood with respect to important reading skills.

“The work we’re doing we call ’Cognitively Based Assessment of, for, and as Learning,’" O’Reilly explained. “Our vision for a future comprehensive K–12 assessment system includes distributing the accountability assessment over multiple administrations throughout the school year, so that the importance of any one assessment and occasion is diminished; tasks can be more authentic and worth teaching to because more time is available for assessment; and the assessments provide interim information while there is still time to take instructional action. Similar to the way teachers assign course grades, accountability information would come from multiple pieces of information, gathered throughout the school year.”

“The next generation of tests must assess deeper skills than our current tests measure,” said Bennett. Over the past 100 years, educational assessment has improved dramatically but those improvements have been almost entirely related to methodology, with little change in the underlying cognitive-scientific basis. ETS played a central role in driving much of that methodological change. It’s time we devoted a similar energy to improving the substantive basis for testing.”

About ETS

At nonprofit ETS, we advance quality and equity in education for people worldwide by creating assessments based on rigorous research. ETS serves individuals, educational institutions and government agencies by providing customized solutions for teacher certification, English language learning, and elementary, secondary and post-secondary education, as well as conducting education research, analysis and policy studies. Founded in 1947, ETS develops, administers and scores more than 50 million tests annually — including the TOEFL® and TOEIC® tests, the GRE® test and The Praxis Series™ assessments — in more than 180 countries, at over 9,000 locations worldwide.