The PIAAC survey assessed the proficiency of adults in three information-processing skills: literacy, numeracy, and problem solving in technology-rich environments. For PIAAC, literacy is defined as "understanding, evaluating, using and engaging with written texts to participate in society, to achieve one's goals, and to develop one's knowledge and potential."13 In the main literacy assessment, texts could be digital (such as Web pages or emails) or print-based (such as newspapers, books, or pamphlets). Regardless of the format, texts could be continuous (such as passages), noncontinuous (such as tables or forms), mixed, or multiple (such as a blog post that contains an initial text and related responses or comments). These texts may naturally appear in work-related, personal, social, community, or educational and training contexts. Survey respondents were required to complete tasks that require application of cognitive strategies such as accessing, identifying, integrating, interpreting, evaluating, or reflecting on information in texts.14
Reading components, introduced for the first time in the 2011 PIAAC survey, are part of the literacy assessment framework, but targeted toward adults near the bottom of the proficiency distribution.15 A detailed account of the PIAAC components assessment framework can be found in Sabatini and Bruce,16 as well as in the reading literacy framework.17 The rationale for component skills rests on the premise that the "meaning construction" processes of reading are built upon a foundation of knowledge of how one's language is represented in one's writing system, that is, component print skills.18 Empirical studies in the reading literature over the past several decades have yielded a rich literature for understanding component reading processes in adults.19 Evidence of an individual's level of print skill can be captured in tasks that can be used to provide evidence of a reader's ability and efficiency in processing the elements of the written language—letters/characters, words, sentences, and larger, continuous text segments.
Reading components are derived from a view of reading literacy skill as a dimension of language proficiency.20 Language learning is typically thought of in four dimensions—reading, writing, speaking, and listening. For native speakers of languages, foundational speaking and listening skills are acquired developmentally. Children acquire a productive/receptive vocabulary of words that we can refer to as their listening/speaking lexicon. Syntax or grammar is also acquired implicitly, allowing individuals to interpret or produce meaningful phrases, sentences, and longer discourse in their native language. As native speakers, we can understand what other speakers are saying in "real time," that is, we process meaning at the speed that the speaker is speaking, which is generally in the range of 150 to 175 words per minute for the average adult English speaker.21
The challenge, in part, in becoming a fluent, skilled reader, is processing printed text as language written down. That is, if a reader can recognize the words rapidly, automatically, and with ease, then the meaning of words can be processed seamlessly, using functional areas of the brain devoted to understanding language, that is, morphology, syntax, semantics, pragmatics, discourse analysis, and semiotics.22 Proficient reading component competencies in any language are characterized by the ease, speed, and minimal attentional resources required when cognitively processing written text, sometimes referred to as automaticity.23 Skilled readers are not typically aware of the spelling, punctuation, or typography of a text; rather their awareness and cognitive effort is applied toward constructing meaning. Thus, an association between reading rate and proficiency is consistently found in studies of adult readers.24
Individuals who struggle with reading, on the other hand, expend considerable cognitive effort and attention in translating print into language, which expends cognitive resources that could better be utilized in constructing, interpreting, and evaluating meaning.25 They struggle with learning to map the writing system (i.e., the printed visual symbols individually and in combination) to the spoken form of the language (i.e., the phonetics, phonology, and morphology).26 Their reading behavior is characterized by slow, effortful processing of text. This can impact not only their recognition of individual words, but also building meaning from sentences and paragraphs of text.27
The irregular mapping of sight-to-sound patterns (and vice versa) in English spelling presents an additional challenge to learning to read fluently.28 When the spelling system of a language is highly regular—which means that with a pronunciation guide, the same spelling pattern routinely maps to the same sounds when spoken—then it is referred to as a transparent orthography.29 German, Finnish, Italian, Spanish, and Swedish are examples of languages with relatively transparent spelling systems. When sight-to-sound (and vice versa) mappings are less consistent, more learning and practice is demanded before fluency is achieved.30
The above account of learning to read focused on native speakers who face the challenge of learning to apply their native language knowledge and skills when processing written texts. For nonnative speakers of a language in which they are reading, the challenge is even greater. All four dimensions of language—speaking, listening, reading, and writing—may need to be learned. In the United States, nonnative speakers of English constitute a sizable and significant proportion of the subpopulation scoring at or below Level 1 in literacy. The PIAAC component tasks cannot be used to distinguish whether the source of challenge for nonnative speakers is their ability to read in English versus more general lack of English language ability; thus, the specific instructional implications may differ. Nonetheless, the components provide an indicator of what nonnative speakers scoring at or below Level 1 can and cannot do when reading in English.
The PIAAC reading components framework covers decoding and word recognition components.31 However, the variations in how a writing system maps to a language differ widely across languages, ranging from alphabetic systems like English and Spanish, to syllabic like the Korean Hangul system, to logographic like the Japanese Kanji system. This cross-language variability led to the decision not to build component tasks for word recognition or decoding in the main PIAAC survey.
Instead, the PIAAC reading components targeted three other levels of reading literacy that correspond to structural levels of language—word meaning, sentence processing, and basic passage comprehension. The word meaning, or print vocabulary,32 tasks targeted the mental lexicon of frequent, concrete words. The sentence processing tasks targeted basic syntactic and semantic knowledge and processing. Finally, the basic passage comprehension tasks targeted discourse processes. Each of these task sets and their corresponding interpretation is discussed more fully in the following sections. The PIAAC component tasks were administered in a paper-based booklet format to all individuals who took the paper-based pathway of the survey.
Recognizing the printed symbols on the page as representing meaningful words is foundational to reading literacy.33 In the print vocabulary task set, the respondent identified everyday words that the average adult speakers of the language would understand if they heard the words spoken aloud. Target words were concrete, imageable nouns of common objects. The items did not include specialized technical or academic words that would only be known by more educated individuals in the population. The words were commonly known across country contexts (e.g., sun, triangle, foot).
Each item in this task set presented an image and four word choices. The respondent had to circle the correct word choice that matched the picture. Figure 1 provides a sample print vocabulary item. Distracters were designed to tap similar semantic and/or orthographic features of the target word. This way, it was less likely that individuals could use only partial knowledge of spelling or visual symbols to guess the correct answer. For example, in the sample item in Figure 1, a reader might guess based on the first sound of the word "ear" that the spelling starts with "e." However, there are two choices that start with "e," making it more challenging to guess.
Figure 1. Sample print vocabulary item.
The sentence is a natural "chunk" when reading continuous text.34 To build meaning from a sentence includes understanding all the words, parsing the syntactic structure, and encoding the propositions in memory. Depending on the specifics of a sentence, other operations might include making anaphoric (e.g., relating pronouns to their referent), causal, or knowledge-based inferences. Thus, each sentence requires some syntactic and semantic processing.
The sentence processing measure presented sentences of increasing difficulty (as indexed by length and density of information) and asked the respondent to make a sensibility judgment about the sentence with respect to general knowledge about the world or about the internal logic of the sentence. For these items, the respondent read the sentence and circled YES if the sentence made sense or NO if the sentence did not make sense. This task demand is consistent with the "evaluation" goal of reading in the PIAAC reading literacy framework.35 Even at the most basic reading level, comprehension or understanding may require evaluating text meaning against one's knowledge of the world, to judge its veracity.36 Figure 2 shows a set of sample sentence processing items.
Figure 2. Sample sentence processing items.
Skilled reading (whether silent or aloud) is rapid, efficient, and fluent. The PIAAC passage comprehension task set targeted silent reading for basic comprehension in multiparagraph prose texts.37 The integration of decoding, word recognition, vocabulary, and sentence processing was required to construct the basic meaning of a short passage. Fluent, efficient performance on a basic, integrated reading task is a building block for handling longer, more complex literacy texts and tasks.
The passage comprehension measure presented four passages, each with embedded items. Passages were constructed based on the kinds of text types that adults typically encounter: narrative, persuasive, and expository. The design used a forced-choice cloze paradigm—that is, a choice was given between a word that correctly completes a sentence in a passage and an option that was incorrect. As the adults read silently through a passage, they would see a word-choice item in selected sentences. The respondent was asked to read the passage and circle the word among the alternatives that made the sentence make sense (in the context of the passage). The incorrect choice was meant to be obviously wrong to a reader with some basic comprehension skills. The incorrect choice could be grammatically or semantically wrong. A sample passage is shown in Figure 3 with the options for selection underlined within the sentences.
Figure 3. Sample passage comprehension items.
Implementation of Reading Components in the PIAAC Survey
The PIAAC survey was administered via computer for most adults, but a subsample of adults was routed to a paper-based pathway. The design and procedures for assigning the computer- versus paper-based instrument to adults were somewhat complex.38 Adults were branched to the paper-based pathway if they lacked any computer experience, failed a core block of basic computer literacy or numeracy skills, or simply opted not to take the survey via computer. Reading components were administered to all adults who took the paper-based pathway, regardless of their literacy proficiency level. Across the entire 23-country PIAAC sample, 91% of the adults who took the paper-based pathway passed the literacy/numeracy core and therefore completed a literacy or numeracy block as well as the reading component tasks. Thirty-one percent of the adults on the paper-based pathway scored at or below Level 1 (versus 15.5% total across the full 23-country sample). Thus, the paper-based subsample had a higher proportion of at or below Level 1 adults than the full, combined sample.
The literacy proficiency scores of adults who took the paper-based pathway are on the same scale as adults who took the computer-based pathway.39 However, the reading components scores could not be put on the general literacy proficiency scale, because the paper-based subsamples were not random subsamples of the full country samples. Consequently, we do not discuss reading components scores in terms of cross-country population estimates, but rather relative patterns of reading component mean scores within and across the common international literacy proficiency score scale levels.
The reading components tasks always were administered after the adults completed the literacy/numeracy core and assessment blocks. The reading components booklet began with simple instructions read to the adult by the survey administrator. The three task sets always appeared in the same order—print vocabulary, sentence processing, and passage comprehension. For the print vocabulary task, the administrator read the directions: "Circle the word that matches the picture you see." Two items appeared on each page. The administrator started the timer when the individual turned the page for the first item, then stopped it when the adult completed the final item.
For the sentence processing task, the administrator read the directions: "Please read each sentence, then circle YES if the sentence makes sense, or circle NO if the sentence does not make sense." Then, the adult was asked to complete three practice items. The 22 items appeared across two printed pages, with 13 on the first page and nine on the second. The administrator started the timer when the adult turned the page for the first item, then stopped it when the individual completed the final item.
For the passage comprehension task, four passages were administered. The administrator read the directions: "Read the following articles. When you come to two words that have been underlined, circle the one word that makes the sentence make sense." The administrator started the timer when the adult turned the page for the first item, then stopped it when the individual completed the final item. The public PIAAC datasets report three separate time estimates, one for each passage (with the last two passages combined into one time, presumably because they were shorter in length).
Table 1 shows the total items per reading components task set, the number of choices for each item in the specific reading components subtest, and the approximate total score for an adult scoring at chance levels. That is, an adult who could not read at all and guessed at every item was likely to receive by chance a total score of about 8 to 9 for print vocabulary, 11 for sentence processing, or 22 for passage comprehension. This should be taken into account when interpreting mean total scores or percentage correct on components for different proficiency levels. Analyses were conducted using IEA Analyzer 3.1.140 and SPSS.41 The software is designed to apply iterations of analyses with plausible values. Weights were applied as appropriate.
TABLE 1. Total items, number of choices, and chance level performance for reading component task sets
Rationale for Country Sample Used in This Report
Unlike when translating the tasks on the main reading literacy survey, translating reading component items across languages may result in different item level difficulty estimates. The relationship between the oral form of the language and its written form may be a determining factor in how easy or difficult it is to learn to read in that language. Some languages may be easier to learn than others, perhaps because the print-to-sound correspondences are highly regular or transparent, making it easier to learn to sound out words once one knows the basic pronunciation key for decoding in that language. This may in turn be reflected in a higher prevalence of adults who show some basic reading skills in one language versus another, because even very basic instruction will yield productive literacy skills. Similarly, differences in syntactic or morphological structures in a language may influence ease or difficulty in interpreting phrases or sentences.
As our main interest in this report is understanding the U.S. adult population, we chose to include English-speaking countries (United States, Canada, United Kingdom,42 and Ireland) as the international comparison reference group. By including English-speaking countries, we can be assured that differences in performance levels of groups are reflective of differences in the population and its educational opportunities, not differences in the relative difficulty of learning to read in the particular language.43 For Canada, which administered the survey in French and English, we included only those who were administered the English-language test.
We also included Italy and Spain in this international comparison, because Italian and Spanish are known to have highly regular sight-to-sound correspondences, which, all things being equal, should make it easier to learn to read in these languages. On the other hand, the populations of these two countries had among the lowest overall performance levels of all 23 countries. Thus, they provide an opportunity to investigate the hypothesis that there is a relative advantage in acquiring foundational reading component skills in these languages even though overall these countries fare less well across the higher proficiency levels in the literacy skills assessed in the main survey.
13 OECD, Literacy, Numeracy and Problem Solving in Technology-Rich Environments: Framework for the OECD Survey of Adult Skills (Paris: OECD Publishing, 2012), 3.
14 For more information on the PIAAC Reading Literacy frameworks, see OECD, 2012, Literacy, Numeracy, or PIAAC Literacy Expert Group, PIAAC Literacy: A Conceptual Framework, OECD Education Working Papers No. 34 (Paris: OECD Publishing, 2009), http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?doclanguage=en&cote=edu/wkp%282009%2913.
15 PIAAC Literacy Expert Group, Conceptual Framework.
16 John P. Sabatini and Kelly Bruce, PIAAC Reading Components: A Conceptual Framework, OECD Education Working Paper No. 33 (Paris: OECD Publishing, 2009), http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?doclanguage=en&cote=edu/wkp%282009%2912.
17 PIAAC Literacy Expert Group, Conceptual Framework.
18 Charles A. Perfetti, Reading Ability (New York: Oxford University Press, 1985); Charles A. Perfetti, "The Universal Grammar of Reading." Scientific Studies of Reading 7, no. 1 (2003): 3-24.
19 Helen Abadzi, Improving Adult Literacy Outcomes: Lessons from Cognitive Research for Developing Countries, World Bank Directions in Development (Washington, DC: World Bank Publications, 2003); Justin Baer, Mark Kutner, and John P. Sabatini, Basic Reading Skills and the Literacy of the America's Least Literate Adults: Results from the 2003 National Assessment of Adult Literacy (NAAL) Supplemental Studies (Washington, DC: National Center for Education Statistics, Institute for Education Sciences, U. S. Department of Education, 2009); Laura C. Bell and Charles A. Perfetti, "Reading Skill: Some Adult Comparisons," Journal of Educational Psychology 86, no. 2 (1994): 244-55;
Anne Cunningham, Keith E. Stanovich, and Mark R. Wilson, "Cognitive Variation in Adult Students Differing in Reading Ability," in Reading and Its Development: Component Skills Approaches, ed. Thomas H. Carr and Betty Ann Levy (New York: Academic Press, 1990), 129-59; Mary E. Curtis, "Development of Components of Reading Skill," Journal of Educational Psychology 72, no. 5 (1980): 656-69; Curtis and Kruidenier, Teaching Adults to Read;
Daphne Greenberg, Linnea C. Ehri, and Dolores Perin, "Are Word-Reading Processes the Same or Different in Adult Literacy Students and Third-Fifth Graders Matched for Reading Level?" Journal of Educational Psychology 89, no. 2 (1997): 262-75; Daphne Greenberg, Linnea C. Ehri, and Dolores Perin, "Do Adult Literacy Students Make the Same Word-Reading and Spelling Errors as Children Matched for Word-Reading Age?" Scientific Studies of Reading 6, no. 3 (2002): 221-43; Charles MacArthur, Timothy R. Konold, Joseph J. Glutting, and Judith A. Alamprese, "Reading Components Skills in Adult Basic Education Learners," Journal of Learning Disabilities (2010);
Daryl F. Mellard, Emily Fall, and Kari. L. Woods, "A Path Analysis of Reading Comprehension for Adults with Low Literacy," Journal of Learning Disabilities 43, no. 2 (2010): 154-65; Alice O. Nanda, Daphne Greenberg, and Robin Morris, "Modeling Child-Based Theoretical Reading Constructs with Struggling Adult Readers," Journal of Learning Disabilities, no. 43 (2010): 139-53; John P. Sabatini, "Efficiency in Word Reading of Adults: Ability Group Comparisons," Scientific Studies of Reading 6, no. 3 (2002): 267-98;
John P. Sabatini, "Word Reading Processes in Adult Learners," in Reading Complex Words: Cross-Language Studies, ed. Egbert M. H. Assink and Dominick Sandra (London: Kluwer Academic, 2003), 265-94; John P. Sabatini, "Learning and Assessment of Adult Reading Literacy," in APA Educational Psychology Handbook: Application to Learning and Teaching, ed. Steven Graham, Karen Harris, and Tim Urdan (Washington DC: American Psychological Association, 2012), 63-83; Elizabeth A. L. Stine-Morrow, Lisa M. Soederberg Miller, and Christopher Hertzog, "Aging and Self-Regulated Language Processing," Psychological Bulletin 132 (2006): 582-606;
John Strucker, Kentaro Yamamoto, and Irwin Kirsch, The Relationship of the Component Skills of Reading to IALS Performance: Tipping Points and Five Classes of Adult Literacy Learners, NCSALL Report No. 29 (Cambridge, MA: National Center for the Study of Adult Learning and Literacy, 2007); Elizabeth L. Tighe and Christopher Schatschneider, "Examining the Relationships of Component Reading Skills to Reading Comprehension in Struggling Adult Readers: A Meta-Analysis," Journal of Learning Disabilities (2014): 1-5.
20 Walter Kintsch, Comprehension: A Paradigm for Cognition (Cambridge, UK: Cambridge University Press, 1998); Perfetti, Universal Grammar.
21 Ronald P. Carver, "The Highly Lawful Relationships among Pseudoword Decoding, Word Identification, Spelling, Listening, and Reading," Scientific Studies of Reading 7, no. 2 (2003): 127-54.
22 Charles A. Perfetti and Donald J. Bolger, "The Brain Might Read That Way," Scientific Studies of Reading 8, no. 3 (2004): 293-304; Keith Rayner, Barbara R. Foorman, Charles A. Perfetti, David Pesetsky, and Mark S. Seidenberg, "How Psychological Science Informs the Teaching of Reading," Psychological Science in the Public Interest 2, no. 2 (2001): 31-74; Rebecca Sandak, W. Einar Mencl, Stephen J. Frost, and Kenneth R. Pugh, "The Neurobiological Basis of Skilled and Impaired Reading: Recent Findings and New Directions," Scientific Studies of Reading 8, no. 3 (2004): 273-92.
23 David LaBerge and S. Jay Samuels, "Toward a Theory of Automatic Information Processing in Reading," Cognitive Psychology 6, no. 2 (1974): 293-323; Alan M. Lesgold and Charles A. Perfetti. "Interactive Processes in Reading Comprehension," Discourse Processes 1, no. 4 (1978): 323-36.
24 Abadzi, Improving Adult Literacy Outcomes; Baer, Kutner, and Sabatini, Basic Reading Skills; Sabatini, "Efficiency in Word Reading."
25 Perfetti, Reading Ability.
26 Perfetti, Universal Grammar.
27 Rayner, Psychological Science.
28 Richard L. Venezky, The American Way of Spelling: The Structure and Origins of American English Orthography (New York: Guilford Press, 1999).
29 Linnea C. Ehri, "Learning to Read Words: Theory, Findings, and Issues," Scientific Studies of Reading 9, no. 2 (2005): 167-88.
30 Karin Landerl and Heinz Wimmer, "Development of Word Reading Fluency and Spelling in a Consistent Orthography: An 8-Year Follow-Up," Journal of Educational Psychology 100 (2008): 150-61; Anniek Vaessen, Daisy Bertrand, Denes Toth, Valeria Csepe, Luıs Faısca, Alexandra Reis, and Leo Blomert, "Cognitive Development of Fluent Word Reading Does Not Qualitatively Differ between Transparent and Opaque Orthographies," Journal of Educational Psychology 102, no. 4 (2010): 827-42.
31 Sabatini and Bruce, PIAAC Reading Components.
32 Hereafter, we will use the term "print vocabulary" for the sake of consistency with previous PIAAC framework documents (e.g., OECD, Literacy, Numeracy; Sabatini and Bruce, PIAAC Reading Components). The full title of this task is "word meaning (print vocabulary)" in those documents. Please note that the term "print" refers to the written word, whether in paper or digital format, and is meant to distinguish from oral.
33 Richard C. Anderson and Peter Freebody, "Vocabulary Knowledge," in Comprehension and Teaching, ed. John T. Guthrie (Newark, DE: International Reading Association, 1981), 77-117; E. D. Hirsch Jr., "Reading Comprehension Requires Knowledge—of Words and the World," American Educator 27, no. 1 (Spring 2003): 10-31; William E. Nagy and Judith A. Scott, "Vocabulary Processes," in Handbook of Reading Research: Volume III, ed. Michael L. Kamil, Peter B. Mosenthal, P. David Pearson, and Rebecca Barr (Mahwah, NJ: Erlbaum, 2000), 269-84; William Nagy and Diane Townsend, "Words as Tools: Learning Academic Vocabulary as Language Acquisition," Reading Research Quarterly 47, no. 1 (2012): 91-108; Gene P. Ouellet, "What's Meaning Got to Do with It: The Role of Vocabulary in Word Reading and Reading Comprehension," Journal of Educational Psychology 98 (2006): 554-66.
34 For example, Joanne F. Carlisle and Melinda Rice, Improving Reading Comprehension: Research-Based Principles and Practices (Baltimore: York Press, 2002); Kintsch, Comprehension.
35 PIAAC Literacy Expert Group, Conceptual Framework.
36 For example, Tobias Richter, "Validation and Comprehension of Text Information: Two Sides of the Same Coin," Discourse Processes 52 (2015): 337-55.
37 Lynn S. Fuchs and Douglas Fuchs, "Identifying a Measure for Monitoring Student Reading Progress," School Psychology Review 21 (1992): 45-58; Miya M. Wayman, Teri Wallace, Hilda Ives Wiley, Renáta Tichá, and Christine A. Espin, "Literature Synthesis on Curriculum-Based Measurement in Reading," Journal of Special Education 41, no. 2 (2007): 85-120.
38 See OECD, OECD Skills Outlook 2013, for a detailed explanation of the design and procedures that governed whether an individual was administered computer- or paper-based instruments.
39 OECD, OECD Skills Outlook 2013.
40 IEA International Database Analyzer, International Association for the Evaluation of Educational Achievement, Hamburg Germany.
41 SPSS Inc., PASW Statistics for Windows, Version 18.0 (Chicago: SPSS Inc., 2009).
42 United Kingdom results refer to England and Northern Ireland only, the two entities reported on for PIAAC. For PIAAC, results for England and Northern Ireland were reported separately at the request of the United Kingdom. For the purposes of this report, such a distinction is unnecessary.
43 Note, however, there will be differences associated with the prevalence of nonnative speaker subpopulations with low skills in a country. Nonnative speakers may face distinct challenges learning to read in the target language of the survey. Native versus nonnative speakers are analyzed later in the report.