skip to main content skip to footer

Building a Validity Argument While Developing and Using an Assessment: A Concurrent Approach for the Winsight Summative Assessment ELA

Stone, Elizabeth; Wylie, E. Caroline
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Winsight, Summative Assessment, Test Quality, Test Validity, Validity Argument, Test Development, Theory of Action, Peer Review, U.S. Department of Education, English Language Arts (ELA), Mathematics, Educational Accountability, Elementary Secondary Education


We describe the summative assessment component within a K–12 assessment program and our development of a validity argument to support its claims with respect to intended uses and interpretations. First, we describe the Winsight assessment program theory of action, a logic model elucidating mechanisms for how use of the assessment components can lead to improved student learning in mathematics and English language arts (ELA) and defining the roles of key groups of stakeholders. We explain our process of developing the validity argument that enumerates sources of validity evidence supporting the Winsight summative assessment claims drawn from the theory of action. We examine candidate structures for the argument, describe the steps in which we unpacked stakeholder‐level claims and defined subclaims to operationalize components, and discuss types and sources of validity evidence to meet the goals of professional and federal guidelines. Finally, we discuss the research agenda and operational analyses that will provide this validity evidence.

Read More