skip to main content skip to footer

Building a Validity Argument While Developing and Using an Assessment: A Concurrent Approach for the Winsight Summative Assessment ELA

Author(s):
Stone, Elizabeth; Wylie, E. Caroline
Publication Year:
2019
Report Number:
RR-19-26
Source:
ETS Research Report
Document Type:
Report
Page Count:
20
Subject/Key Words:
Winsight, Summative Assessment, Test Quality, Test Validity, Validity Argument, Test Development, Theory of Action, Peer Review, U.S. Department of Education, English Language Arts (ELA), Mathematics, Educational Accountability, Elementary Secondary Education

Abstract

We describe the summative assessment component within a K–12 assessment program and our development of a validity argument to support its claims with respect to intended uses and interpretations. First, we describe the Winsight assessment program theory of action, a logic model elucidating mechanisms for how use of the assessment components can lead to improved student learning in mathematics and English language arts (ELA) and defining the roles of key groups of stakeholders. We explain our process of developing the validity argument that enumerates sources of validity evidence supporting the Winsight summative assessment claims drawn from the theory of action. We examine candidate structures for the argument, describe the steps in which we unpacked stakeholder‐level claims and defined subclaims to operationalize components, and discuss types and sources of validity evidence to meet the goals of professional and federal guidelines. Finally, we discuss the research agenda and operational analyses that will provide this validity evidence.

Read More