Human Capital and International Large-Scale Assessment

Understanding the Basic Reading Skills of U.S. Adults: Reading Components in the PIAAC Literacy Survey
John Sabatini

To be able to learn from text is perhaps the most important foundational skill for U.S. adult citizens' health, well-being, and social and economic advancement. It is not only an essential skill when competing for jobs in the 21st century workforce, but it is also a gateway to lifelong learning, education and training. View the full report >

Development of an Evidence-Based Reading Fluency Program for Adult Literacy Learners
J. R. Shore, J. P. Sabatini, J. Lentini, S. Holtzman, & A. McNeil
Reading Psychology, Vol. 36, pp. 86–104, 2015

The authors discuss the role of fluency as a key component of adults' reading skills. Fluency has not received sufficient attention from research. The authors describe an evidence-based adult Guided Repeated Reading program for low-literate adult learners and discuss the future impact of its implementation. View citation record >

America's Skills Challenge: Millennials and the Future
M. Goodman, A. Sands & R. Coley
ETS report

The report uses data from the PIAAC survey to compare the United States to 21 other member countries in the OECD, focusing on "millennials," i.e., young adults born after 1980 who were 16–34 years of age at the time of the assessment. On average, America's millennials demonstrated weak skills in literacy, numeracy and problem-solving in technology-rich environments compared to their international peers. View citation record >

Handbook of International Large-Scale Assessment: Background, Technical Issues, and Methods of Data Analysis
Editors: L. Rutkowski, M. von Davier, & D. Rutkowski
Publisher: CRC Press

This book highlights the importance of international large-scale assessment data to policy and research. It reviews methodological aspects and features of the studies based on operational considerations, analytics and reporting. Chapter authors describe methods of interest to readers who have a good grounding in quantitative methods, but who are not necessarily quantitative methodologists. View citation record >

The Role of International Large-Scale Assessments: Perspectives from Technology, Economy, and Educational Research
Editors: M. von Davier, E. Gonzalez, I. Kirsch, & K. Yamamoto
Publisher: Springer

This ETS-edited volume compiles interdisciplinary papers on research using international large-scale assessment data. It includes perspectives from economics, policy, education, civics education and educational technology. Chapter authors discuss the challenges that come with the increasing interest and the broadening scope of international large-scale assessments. View publisher's page >

Technical Report of the Survey of Adult Skills (PIAAC)
Publisher: Organisation for Economic Co-operation and Development (OECD)

This report, with contributions from ETS and other members of the consortium involved with PIAAC, describes in detail the design, development and data analysis used for the first computer-delivered large-scale assessment of adult literacy. View full report >

Fault Lines in Our Democracy: Civic Knowledge, Voting Behavior, and Civic Engagement in the United States
R. Coley & A. Sum
ETS Policy Information Center Report

The report examines gaps in educational attainment and income among different segments of the U.S. population with regards to civic knowledge, voting behavior and civic engagement. View citation record >

An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment
H. Braun, I. Kirsch, & K. Yamamoto
Teachers College Record, Vol. 113, No. 11, pp. 2309–2344

This article reports on a study of monetary incentives for 12th-grade students taking an assessment designed to replicate the NAEP reading test. The study concluded that NAEP may both underestimate the reading abilities of students enrolled in 12th grade and yield biased estimates of certain achievement gaps. View publisher's page >

Measuring Growth in a Longitudinal Large Scale Assessment with a General Latent Variable Model
M. von Davier, X. Xu, & C. H. Carstensen
Psychometrika, Vol. 76, No. 2, pp. 318–336

This article presents approaches for use with longitudinal large-scale assessment data, i.e., data that were repeatedly collected on the same sample of students or adults. The article describes models that made it possible to examine whether the Matthew-effect (those who have more, gain more) can be identified in a national extension of the PISA assessment. View citation record >

Investigation of Model Fit and Score Scale Comparability in International Assessments
M. E. Oliveri & M. von Davier
Psychological Test and Assessment Modeling, Vol. 53, No. 3, pp. 315–333

This article presents evidence on scale stability and the need to account for national variations in how some tasks in international assessments are understood. While rigorous translation, verification and assessments of cultural appropriateness reduce the threat of cross-country differences in how tasks are understood, it is always possible to detect deviations from the international standard on a limited number of tasks. This article presents an approach of how such deviation can be remedied. View article >

Relationships Among Reading Skills of Adults with Low Literacy
J. P. Sabatini, Y. Sawaki, J. R. Shore, & H. S. Scarborough
Journal of Learning Disabilities, Vol. 43, No. 2, pp. 122–138

In this study, researchers used a method known as confirmatory factor analysis to examine the relationship between literacy and several component reading skills (e.g., word recognition, fluency, language comprehension and vocabulary skills) in adults. View publisher's page >

Basic Reading Skills and the Literacy of America's Least Literate Adults: Results from the 2003 National Assessment of Adult Literacy (NAAL) Supplemental Studies
J. Baer, M. Kutner, & J. Sabatini
National Center For Education Statistics Report No. NCES 2009481

The 2003 NAAL assessed the English literacy skills of U.S. adults. This report focuses on results from the supplemental basic reading skills measures given to all adults in the survey, as well as a special battery of assessment given to the least literate subpopulation. View citation record >

What are Plausible Values and Why Are They Useful?
M. von Davier, E. Gonzalez, & R. J. Mislevy
IERI Monograph Series: Issues and Methodologies in Large Scale Assessments, Vol. 2.

This report describes one of the fundamental tools developed to enable secondary analysis of large-scale assessment data through plausible values, a form of imputations that incorporates all available information into proficiency values. The report provides a less technical and more practical overview of the use of this methodology and contracts appropriate analysis with shortcuts that lead to biased analysis results. View the article >

Preparing for an Epidemic of Limited Health Literacy: Weathering the Perfect Storm
R. M. Parker, M. S. Wolf, & I. Kirsch
Journal of General Internal Medicine, Vol. 23, No. 8, pp. 1273–1276

Research has demonstrated strong links between literacy and health. ETS's recent report America's Perfect Storm considers the great risks the United States faces as a result of declining adult literacy, shifting demographics and a changing economy. It is essential to understand how these changes will impact health care. View publisher's page >

America's Perfect Storm: Three Forces Changing Our Nation's Future
I. Kirsch, H. Braun, K. Yamamoto, & A. Sum
ETS Policy Information Report

This report looks at the convergence of three powerful sociological and economical forces that are changing the future of the United States: substantial disparities in skill levels (reading and math), seismic economic changes (widening wage gaps), sweeping demographic shifts (less education, lower skills). View summary >

Adult Learners in a Changing America
I. Kirsch, M. Lennon, & C. Tamassia
in B. Guzzetti (Editor), Literacy for the New Millennium, Vol. 4, pp. 91–122 Publisher: Praeger

This ETS-authored article appears in a multivolume series on literacy research. View citation record >

Learning a Living: First Results of the Adult Literacy and Life Skills Survey
Organisation for Economic Co-operation and Development & Statistics Canada

This report on the Adult Literacy and Life Skills survey (ALL) presents evidence on the nature and magnitude of the literacy gaps faced by OECD countries and how these gaps evolved over the previous decade. It contains interpretation of data for the seven countries participating in the first round of data collection. View the report here >

A Human Capital Concern: The Literacy Proficiency of United States Immigrants
A. Sum, I. Kirsch, & K. Yamamoto
ETS Policy Information Report

In this report, the authors characterize the literacy proficiencies of U.S. residents born abroad and compares their performance not only with adults born in the U.S., but also with their foreign-born counterparts in other high-income countries around the world. View citation record >

Pathways to Labor Market Success: The Literacy Proficiency of United States Adults
A. Sum I. Kirsch, & K. Yamamoto
ETS Policy Information Report

The report draws upon background and assessment data and information collected from the National Adult Literacy Survey (NALS) and the International Adult Literacy Survey (IALS). The authors find connections between the literacy skills of adults and their success in the labor market during the 1990s. View citation record >

Digital Transformation: A Framework for ICT Literacy
International ICT Literacy Panel (2002)

This report describes the work of a panel that ETS convened in January 2001 to study the growing importance of existing and emerging information and communication technologies (ICT) and their relationship to literacy. The panel was made up of international experts from education, government, nongovernmental organizations (NGOs), labor and the private sector. View citation record >

Reading for Change: Performance and Engagement Across Countries
I. Kirsch et al. (2002)
Publisher: Organisation for Economic Co-operation and Development

This report looks closely at the reading results of the first Program in International Student Assessment (PISA) survey. It shows reading literacy levels of students age 15, in 32 countries, in terms of how well they can use written materials to meet the challenges of the real world and to become lifelong learners. View citation record >

Literacy in the Information Age: Final Report of the International Adult Literacy Survey
Organisation for Economic Co-operation and Development and Statistics Canada (2000)

This report looks at the literacy gaps faced by 20 OECD countries for which comparable household assessment results are included. It offers insights into the factors that influence the development of adult skills in various settings — at home, at work and across countries. View the report

Adult Literacy in OECD Countries: Technical Report on the First International Adult Literacy Survey
T. S. Murray, I. S. Kirsch, and L. Jenkins (Eds.) (1997)
National Center for Education Statistics

This publication provides detail on IALS, which extended educational measurement and assessment to the adult population. It describes detail on the design, development and statistical analysis of this assessment, conducted in eight countries. View publisher's page >