As societies become increasingly advanced, adults are expected to use information in more complex ways and to maintain and enhance their skills throughout their lifetimes. Such skills are critical not only for personal achievement, but also for positive social, educational and economic outcomes around the world.
ETS's research in this area is based on international large-scale comparative surveys of both adults and school-age youth. The focus is on core skills such as reading, math/numeracy and science literacy. Other areas addressed in these surveys include innovative constructs such as financial literacy, collaborative problem solving and problem solving in technology-rich environments.
ETS's research in human capital and international large-scale assessment is conducted both in the United States and around the world. Our large-scale international comparative surveys are designed to meet policy needs reflecting a growing interest in skills as human capital. These are some examples:
OECD Survey of Adult Skills
ETS manages this major assessment for the Organisation for Economic Co-operation and Development (OECD) as part of the Programme for the International Assessment of Adult Competencies (PIAAC). This is the first large-scale, computer-delivered comparative assessment of adults in the world. It provides policymakers with a profile of adults' knowledge, skills and competencies in literacy, numeracy, and problem solving in technology-rich environments. PIAAC expands upon earlier literacy assessments, and is the first multistage adaptive test used in large-scale comparative assessment.
We also work on two other projects related to the PIAAC program:
- Education and Skills Online — A new computer-delivered assessment linked to PIAAC. It provides individual results for persons who want to demonstrate readiness skills, and allows institutions, organizations and local governments to assess a population for training and research purposes.
- The World Bank's Skills Toward Employment and Productivity (STEP) — This program, which is linked to PIAAC, measures reading literacy and component skills, which include vocabulary knowledge, sentence processing and passive comprehension. It seeks to help countries assess how different skill sets affect labor-market opportunities, and provides information on employability and marketability.
In November 2013, ETS organized the PIAAC Invitational Research Conference, which highlighted the importance of the PIAAC results for understanding the role of skills in a global context. Watch the conference's opening video.
Programme for International Student Assessment (PISA)
ETS manages and develops this global evaluation of education systems for the OECD. PISA is a multinational survey of 15-year-old students, currently administered in some 70 countries. The test, which transitioned from paper to computer in 2015, assesses what students know and their ability to apply their knowledge in real-life situations at an age when they begin to fully participate in society.
TIMSS and PIRLS
ETS acts as consultant to Boston College on two international assessments: Trends in International Mathematics and Science Study, as well as the Progress in International Reading Literacy Study.
IEA-ETS Research Institute (IERI)
ETS collaborates with the International Association for the Evaluation of Educational Achievement (IEA) on research studies, dissemination efforts, and professional development focused on improving the science of large-scale assessments. IERI publishes an international journal and runs workshops on methodology using existing large-scale assessment databases. Both independently and as part of its involvement in IERI, ETS's researchers also produce research reports, books and book chapters related to human capital and international large-scale assessment (see list below).
International Large-Scale Assessment Conference
Designed to present and discuss multidisciplinary issues related to the use and implementation of international large-scale assessments, the 2011 event informed funders, policymakers, managers and technical staff of existing international large-scale assessment programs.
Read the publication that resulted from the conference: The Role of International Large-Scale Assessments: Perspectives from Technology, Economy, and Educational Research.
Here is a selection of significant publications related to human capital and international large-scale assessment:
Choosing Our Future: A Story of Opportunity in America
I. S. Kirsch, H. Braun, M. L. Lennon, & A. Sands
An ETS report
Over the past 40 years, technological changes, interactions among powerful global economic forces, government policies and business practices have generated a self-sustaining set of dynamics that have altered the workplace, the demands of everyday life, and our civic and social lives. The authors analyze the complex and self-sustaining dynamics governing the distribution and intergenerational transmission of opportunity with the goal of advancing our national conversation about the increasing polarization of opportunity and why we must take action.
The Dynamics of Opportunity in America: Evidence and Perspectives
Editors: I. Kirsch & H. Braun
Our children are beginning life from very different starting points that place them on distinctly different trajectories of growth and development, leading to vastly different adult outcomes. Leaders from fields including education, economics, demography and political science illuminate these dynamics that are converging to create a cycle of opportunity inequality in the United States, and offer suggestions for reversing these harmful trends.
To be able to learn from text is perhaps the most important foundational skill for U.S. adult citizens' health, well-being, and social and economic advancement. It is not only an essential skill when competing for jobs in the 21st century workforce, but it is also a gateway to lifelong learning, education and training. View the full report >
Development of an Evidence-Based Reading Fluency Program for Adult Literacy Learners
J. R. Shore, J. P. Sabatini, J. Lentini, S. Holtzman, & A. McNeil
Reading Psychology, Vol. 36, pp. 86–104, 2015
The authors discuss the role of fluency as a key component of adults' reading skills. Fluency has not received sufficient attention from research. The authors describe an evidence-based adult Guided Repeated Reading program for low-literate adult learners and discuss the future impact of its implementation. View citation record >
America's Skills Challenge: Millennials and the Future
M. Goodman, A. Sands & R. Coley
The report uses data from the PIAAC survey to compare the United States to 21 other member countries in the OECD, focusing on "millennials," i.e., young adults born after 1980 who were 16–34 years of age at the time of the assessment. On average, America's millennials demonstrated weak skills in literacy, numeracy and problem-solving in technology-rich environments compared to their international peers. View citation record >
Handbook of International Large-Scale Assessment: Background, Technical Issues, and Methods of Data Analysis
Editors: L. Rutkowski, M. von Davier, & D. Rutkowski
Publisher: CRC Press
This book highlights the importance of international large-scale assessment data to policy and research. It reviews methodological aspects and features of the studies based on operational considerations, analytics and reporting. Chapter authors describe methods of interest to readers who have a good grounding in quantitative methods, but who are not necessarily quantitative methodologists. View citation record >
The Role of International Large-Scale Assessments: Perspectives from Technology, Economy, and Educational Research
Editors: M. von Davier, E. Gonzalez, I. Kirsch, & K. Yamamoto
This ETS-edited volume compiles interdisciplinary papers on research using international large-scale assessment data. It includes perspectives from economics, policy, education, civics education and educational technology. Chapter authors discuss the challenges that come with the increasing interest and the broadening scope of international large-scale assessments. View publisher's page >
Technical Report of the Survey of Adult Skills (PIAAC)
Publisher: Organisation for Economic Co-operation and Development (OECD)
This report, with contributions from ETS and other members of the consortium involved with PIAAC, describes in detail the design, development and data analysis used for the first computer-delivered large-scale assessment of adult literacy. View full report >
Fault Lines in Our Democracy: Civic Knowledge, Voting Behavior, and Civic Engagement in the United States
R. Coley & A. Sum
ETS Policy Information Center Report
The report examines gaps in educational attainment and income among different segments of the U.S. population with regards to civic knowledge, voting behavior and civic engagement. View citation record >
An Experimental Study of the Effects of Monetary Incentives on Performance on the 12th-Grade NAEP Reading Assessment
H. Braun, I. Kirsch, & K. Yamamoto
Teachers College Record, Vol. 113, No. 11, pp. 2309–2344
This article reports on a study of monetary incentives for 12th-grade students taking an assessment designed to replicate the NAEP reading test. The study concluded that NAEP may both underestimate the reading abilities of students enrolled in 12th grade and yield biased estimates of certain achievement gaps. View publisher's page >
Measuring Growth in a Longitudinal Large Scale Assessment with a General Latent Variable Model
M. von Davier, X. Xu, & C. H. Carstensen
Psychometrika, Vol. 76, No. 2, pp. 318–336
This article presents approaches for use with longitudinal large-scale assessment data, i.e., data that were repeatedly collected on the same sample of students or adults. The article describes models that made it possible to examine whether the Matthew-effect (those who have more, gain more) can be identified in a national extension of the PISA assessment. View citation record >
Investigation of Model Fit and Score Scale Comparability in International Assessments
M. E. Oliveri & M. von Davier
Psychological Test and Assessment Modeling, Vol. 53, No. 3, pp. 315–333
This article presents evidence on scale stability and the need to account for national variations in how some tasks in international assessments are understood. While rigorous translation, verification and assessments of cultural appropriateness reduce the threat of cross-country differences in how tasks are understood, it is always possible to detect deviations from the international standard on a limited number of tasks. This article presents an approach of how such deviation can be remedied. View article >
Relationships Among Reading Skills of Adults with Low Literacy
J. P. Sabatini, Y. Sawaki, J. R. Shore, & H. S. Scarborough
Journal of Learning Disabilities, Vol. 43, No. 2, pp. 122–138
In this study, researchers used a method known as confirmatory factor analysis to examine the relationship between literacy and several component reading skills (e.g., word recognition, fluency, language comprehension and vocabulary skills) in adults. View publisher's page >
Basic Reading Skills and the Literacy of America's Least Literate Adults: Results from the 2003 National Assessment of Adult Literacy (NAAL) Supplemental Studies
J. Baer, M. Kutner, & J. Sabatini
National Center For Education Statistics Report No. NCES 2009481
The 2003 NAAL assessed the English literacy skills of U.S. adults. This report focuses on results from the supplemental basic reading skills measures given to all adults in the survey, as well as a special battery of assessment given to the least literate subpopulation. View citation record >
What are Plausible Values and Why Are They Useful?
M. von Davier, E. Gonzalez, & R. J. Mislevy
IERI Monograph Series: Issues and Methodologies in Large Scale Assessments, Vol. 2.
This report describes one of the fundamental tools developed to enable secondary analysis of large-scale assessment data through plausible values, a form of imputations that incorporates all available information into proficiency values. The report provides a less technical and more practical overview of the use of this methodology and contracts appropriate analysis with shortcuts that lead to biased analysis results. View the article >
Preparing for an Epidemic of Limited Health Literacy: Weathering the Perfect Storm
R. M. Parker, M. S. Wolf, & I. Kirsch
Journal of General Internal Medicine, Vol. 23, No. 8, pp. 1273–1276
Research has demonstrated strong links between literacy and health. ETS's recent report America's Perfect Storm considers the great risks the United States faces as a result of declining adult literacy, shifting demographics and a changing economy. It is essential to understand how these changes will impact health care. View publisher's page >
America's Perfect Storm: Three Forces Changing Our Nation's Future
I. Kirsch, H. Braun, K. Yamamoto, & A. Sum
ETS Policy Information Report
This report looks at the convergence of three powerful sociological and economical forces that are changing the future of the United States: substantial disparities in skill levels (reading and math), seismic economic changes (widening wage gaps), sweeping demographic shifts (less education, lower skills). View summary >
Adult Learners in a Changing America
I. Kirsch, M. Lennon, & C. Tamassia
in B. Guzzetti (Editor), Literacy for the New Millennium, Vol. 4, pp. 91–122 Publisher: Praeger
This ETS-authored article appears in a multivolume series on literacy research. View citation record >
Learning a Living: First Results of the Adult Literacy and Life Skills Survey
Organisation for Economic Co-operation and Development & Statistics Canada
This report on the Adult Literacy and Life Skills survey (ALL) presents evidence on the nature and magnitude of the literacy gaps faced by OECD countries and how these gaps evolved over the previous decade. It contains interpretation of data for the seven countries participating in the first round of data collection. View the report here >
A Human Capital Concern: The Literacy Proficiency of United States Immigrants
A. Sum, I. Kirsch, & K. Yamamoto
ETS Policy Information Report
In this report, the authors characterize the literacy proficiencies of U.S. residents born abroad and compares their performance not only with adults born in the U.S., but also with their foreign-born counterparts in other high-income countries around the world. View citation record >
Pathways to Labor Market Success: The Literacy Proficiency of United States Adults
A. Sum I. Kirsch, & K. Yamamoto
ETS Policy Information Report
The report draws upon background and assessment data and information collected from the National Adult Literacy Survey (NALS) and the International Adult Literacy Survey (IALS). The authors find connections between the literacy skills of adults and their success in the labor market during the 1990s. View citation record >
Digital Transformation: A Framework for ICT Literacy
International ICT Literacy Panel (2002)
This report describes the work of a panel that ETS convened in January 2001 to study the growing importance of existing and emerging information and communication technologies (ICT) and their relationship to literacy. The panel was made up of international experts from education, government, nongovernmental organizations (NGOs), labor and the private sector. View citation record >
Reading for Change: Performance and Engagement Across Countries
I. Kirsch et al. (2002)
Publisher: Organisation for Economic Co-operation and Development
This report looks closely at the reading results of the first Program in International Student Assessment (PISA) survey. It shows reading literacy levels of students age 15, in 32 countries, in terms of how well they can use written materials to meet the challenges of the real world and to become lifelong learners. View citation record >
Literacy in the Information Age: Final Report of the International Adult Literacy Survey
Organisation for Economic Co-operation and Development and Statistics Canada (2000)
This report looks at the literacy gaps faced by 20 OECD countries for which comparable household assessment results are included. It offers insights into the factors that influence the development of adult skills in various settings — at home, at work and across countries. View the report
Adult Literacy in OECD Countries: Technical Report on the First International Adult Literacy Survey
T. S. Murray, I. S. Kirsch, and L. Jenkins (Eds.) (1997)
National Center for Education Statistics
This publication provides detail on IALS, which extended educational measurement and assessment to the adult population. It describes detail on the design, development and statistical analysis of this assessment, conducted in eight countries. View publisher's page >
Find a Publication
Retooling Literacy Education for the Twenty-First Century: Key Findings of the Reading for Understanding Initiative and Their Implications
An ETS report.
An ETS report.
An ETS report.