As a first step, we estimated the prevalence of adults with low reading literacy skill levels on the PIAAC general literacy scale for each country. Table 2 shows the estimated percentage of adults in a country by proficiency level for the six countries analyzed in this report (Sample), as well as for the 23 countries (International) in the full (computer- plus paper-based samples) PIAAC sample. In the subsample of six countries, the estimated size of the population below Level 1 or at Level 1 are 4.7 and 15.8, respectively, with Spain and Italy showing larger percentages of their respective populations at or below Level 1. Note that the six-country subsample (Sample) has a higher average percentage of adults in comparison to the 23-country full PIAAC sample (International) both at Below Level 1 (4.7 versus 3.3) and at Level 1 (15.8 versus 12.2). Table 3 shows the same information but for the paper-based subsamples only. Note the higher percentage of adults at lower proficiency levels in the paper-based pathway only in comparison to the full PIAAC sample. This shows that the procedures for routing adults to the paper-based pathway were somewhat effective in directing more adults with lower literacy proficiencies toward the paper-based instruments. Nonetheless, about 60% of the Sample group has proficiency levels of Level 2 or higher.
In the remainder of the report, we will only report analyses using the six-country, paper-based pathway subsample (Sample) to facilitate comparison among the English-speaking countries, along with Italy and Spain. In subsequent tables and figures, we continue to position the Sample mean after the U.S. and Canada samples to facilitate visual interpretation, because this is the position where the subsample mean generally falls. Thus, the reader can estimate the magnitude of difference between the U.S./Canada and the Sample means.
TABLE 2. Percentages of population by literacy proficiency level (paper and computer pathways)
TABLE 3. Percentages of population by literacy proficiency level (paper pathway only)
Reading Component Skill Performance Accuracy
Tables 4-7 and Figures 4-5 display the average percentage correct scores for each of the reading components task sets and the sum across all three task sets by proficiency level. As one can see, there was a significant increase for each of the reading components (print vocabulary, sentence processing, and passage comprehension) as reading literacy proficiency level increased. One can also observe that the trajectory on reading components scores reaches an asymptote for adults with proficiency levels 3 and above. Mean performance at these higher levels of literacy proficiency is generally 95% or higher on each task set. In the remainder of the report, we will only show results up to Level 3 for simplicity of presentation.
It is somewhat encouraging that the mean performance on print vocabulary (Table 4), even for the U.S. adults below Level 1, was well above chance levels at 77% correct. However, the U.S. means were lower than English international counterparts, and also lower than Italy and Spain, both at and below Level 1.
TABLE 4. Percentage correct by literacy proficiency level for print vocabulary
For sentence processing (Table 5), U.S. adults were at chance levels of performance, while their English counterparts' mean percentage correct ranged from 64% to 71%. U.S. adults at Level 1 also had lower means than their international counterparts. There continued to be a mean difference in sentence processing tasks even at Level 2, with U.S. adults only achieving parity with other countries at Level 3 or above. Note that Italy and Spain showed higher average levels of performance than the English-speaking countries.
TABLE 5. Percentage correct by literacy proficiency level for sentence processing
For passage comprehension, Below Level 1 U.S. adults were again at chance levels of performance, while their English counterparts' mean percentage correct ranged from 59% to 69% (Table 6). U.S. adults at Level 1 also showed mean differences in comparison to international counterparts. In general, there was a drop in relative performance for passage versus sentence for Below Level 1 adults across countries. This mean difference in passage reading attenuated some at Level 1 and more so at Level 2.
One may have noticed that sentence and passage reading means were closely aligned across the higher levels of literacy proficiency, with passage means sometimes higher than sentence means toward the higher proficiency levels. This is because the most difficult sentence items are typically more difficult than any of the passage items. Thus, even adults who are relatively more proficient may still make errors on these challenging sentence items while likely finding all passage items relatively easy to answer.
TABLE 6. Percentage correct by literacy proficiency level for passage comprehension
Table 7 shows the percentage correct summing all three component task set scores (total items = 100). The results showed the same general pattern, confirming the relatively lower performance levels at and below Level 1 for U.S. adults in comparison to the other countries in this sample.
TABLE 7. Percentage correct for sum of all three reading component task sets
Figures 4 and 5 graphically show the percentage correct means for each reading component by reading literacy proficiency levels. The scale range is the same on all three charts for ease of evaluating the magnitude of difference in performance across the component tasks. Recall that chance performance on the print vocabulary measure is about 25%. In general, the U.S. adults at and below Level 1 had lower scores on each of the reading components than the other groups in the sample, with English-speaking Canada scoring next lowest in all comparisons. Italy and Spain, despite larger percentages of their populations scoring at or below Level 1 overall, trended toward relatively higher reading component scores than the English-speaking countries in print vocabulary, reflecting perhaps that acquiring basic word reading skill is relatively easier in the languages of these countries because of strong correspondence of the written to spoken language. This advantage was not found in the sentence processing or passage comprehension task sets. Future research will need to be conducted to test alternate explanations for this pattern.
Figure 4. Charts showing country percentage correct on print vocabulary, sentence processing, and passage comprehension by literacy proficiency levels.
Figure 5. Chart showing country percentage correct for sum of all three component skill task sets by literacy proficiency levels.
Reading Component Skill Performance: Time to Complete Task Sets
Next, we computed the average time (in minutes) to complete each of the reading components task sets and the total time summed across all three task sets (see Tables 8-11).44 As one can see, there was a significant decrease in time to complete each of the reading components (print vocabulary, sentence processing, and passage comprehension), as reading literacy proficiency level increased. Unlike percentage correct scores, which reached an asymptote for adults at proficiency Level 3 and above, the mean rate of response continued to decrease across the entire ability distribution (not shown). This is to be expected. As reading component skills become more accurate, rate of processing text becomes more efficient.
TABLE 8. Time (in minutes) by literacy proficiency level to complete print vocabulary task set
TABLE 9. Time (in minutes) by literacy proficiency level to complete sentence processing task set
TABLE 10. Time (in minutes) by literacy proficiency level to complete passage comprehension task set
TABLE 11. Time (in minutes) by literacy proficiency level to complete reading component task set
Unlike the percentage correct data, the U.S. results for timing data were not so different from other countries in the sample. What is important to note is the magnitude of the increase in time to complete tasks for the Below Level 1 and at Level 1 groups. Skilled adults read continuous English prose text at an average rate about 200 to 250 words per minute with good comprehension.45 Readers may slow down their rate when the text is highly complex, they are unfamiliar with the topic, they are trying to learn or study, or they need to think critically about a question or problem related to the text.46 However, given the difficulty level of the reading component tasks, a good reader would not need to slow down very much. Hence, the reading components are an index of reading ease, automaticity, and fluency. In fact, the sentence task format is very similar to other tests used as indicators of a reading fluency construct.47
If we set Level 3 as a reference point for a typical, skilled adult reader, then we can estimate the relative additional time required for readers at lower proficiency levels to complete the task set. Table 12 shows these ratios. As the table indicates, the sample country mean at Level 2 was about 20% (country range 14% to 27%) slower than the Level 3 group mean. This increased to an average of 42% slower at Level 1 (country range 33% to 57%) and 64% at Below Level 1 (country range 46% to 75%).
TABLE 12. Ratio of time to complete reading component task set with Level 3 as reference time (denominator)
Native and Nonnative Speakers
How did native versus nonnative speakers of the test language perform on reading components task sets?48 Figure 6 compares native (left column) versus nonnative (right column) speaker groups for print vocabulary, sentence processing, and passage comprehension by literacy proficiency levels. These charts show that the U.S. mean difference in reading components was strongly associated with the low performance of nonnative English speakers in the United States. When comparing only native speaking groups (left column) across the country sample, the U.S. Below Level 1 group had scores comparable to the international sample average in print vocabulary (92% to 90%), still relatively lower in sentence processing (72% to 76%), and much lower in passage comprehension (66% to 76%). At Level 1, the same pattern of results was evident.
However, when one examined nonnative speaker performance (right column), the U.S. difference increased dramatically in comparison to the other country samples. Below Level 1, nonnative speakers of English in the United States had average percentage correct scores of 68% on print vocabulary, 39% on sentence processing, and 35% on passage comprehension.
Figure 6. Charts showing country percentage correct for native and nonnative speakers of a language for print vocabulary, sentence processing, and passage comprehension by literacy proficiency levels.
The score difference between those with computer experience versus those without was largest for those scoring below Level 1 (Figure 7), with the mean difference closing as proficiency increased. Table 13 shows how the difference between those with and without self-reported computer experience varied with proficiency level. As Table 13 shows, the mean difference diminished with proficiency. The U.S. difference was largest in the sample, and this difference was relatively greater than other countries across each proficiency level.
Figure 7. Computer experience by country group mean differences on total for all reading components by proficiency level for Yes versus No (computer experience).
TABLE 13. Mean difference in total reading component score by computer experience (Yes-No experience)
Relations among Reading Components and General Reading Literacy Proficiency
To further understand the relations between reading components and proficiency, we computed correlations among the variables for the six-country subsample. The correlations were computed using the entire subsample that took the paper-based assessment pathway, not just adults at or below Level 1. As one can see in Tables 14-15, there were moderate to strong correlations among the reading component scores, and small to moderate correlations to literacy scores for adults across the entire distribution for both the United States and six-country sample proficiency levels. However, the relationships were strongest for the U.S. population. Stronger correlations of the components to proficiency is consistent with an interpretation that weak component skills are continuing to impact overall literacy proficiency performance.
The tables also show that the times to complete task sets were negatively correlated, although the magnitude of the correlations were smaller than were the reading component accuracy scores. A negative correlation shows that as individuals are higher on the proficiency scale, their time decreases—evidence of fluency and automaticity. In part, this weaker correlation may be a consequence of error associated with the measurement of time using the paper-based administration procedures. However, as we shall see in the next analyses, total time is a contributor to the overall relationship to reading literacy proficiency scores, even after accounting for reading component accuracy scores.
TABLE 14. Correlations among literacy proficiency score, reading components accuracy, and time to complete reading component task sets for U.S. sample
TABLE 15. Correlations among literacy proficiency score, reading components accuracy, and time to complete reading component task sets for six-country sample
Another way to understand the relation of reading components to overall proficiency is to look at how strongly the components are related to literacy proficiency scores. The regression models in Table 16 were computed using the entire six-country sample that took the paper-based assessment pathway, not just adults at or below Level 1. That is, we included all adults who were administered reading components regardless of predicted literacy proficiency score. Because accuracy scores reached an asymptote around Level 3, we can infer that reading components accuracy scores predict literacy scores most strongly for adults below this level. This assumption will not necessarily hold for the timing data, as we observed that there was a decrease in average time across the entire ability distribution.
With literacy proficiency score as the dependent variable, Table 16 shows variance accounted for by print vocabulary (PV), sentence processing (SP), and passage comprehension (PC), the combination of the three, and finally with timing information included. Print vocabulary accuracy showed the weakest relation to literacy scores. This most likely reflects the near-ceiling performance levels at and beyond Level 2. Sentence processing and passage comprehension, on the other hand, showed a modest but significant relationship across the ability range.
Adding the variance of vocabulary and sentence together (column PV+SP) added a little to the variance, and adding the passage scores (column PV+SP+PC) increased the strength of the relationship further. Finally, including the time to complete information (column All RC+Timer) made a large, significant contribution to the relationship, with an international average of 30% of the variance accounted for by the component skill scores. Note also that there was relatively little between-country variation in the relation between components and literacy scores, except for a stronger relationship in the U.S. sample. As noted previously, stronger associations of the components to proficiency scores is consistent with an interpretation that weak component skills are continuing to impact overall proficiency for adults, particularly in the U.S. sample.
TABLE 16. Multiple regression model showing adjusted R-squared values with literacy proficiency scores as dependent variable and combinations of reading components (RC) scores and time to complete reading component tasks as independent variables
44 We omitted times that were less than 20 seconds for any single component task. We also reset all times that were beyond 2.5 standard deviations of the mean of each reading component task set to the value at 2.5 standard deviations. This reduced the extreme values at either end of the timing distribution.
45 Keith Rayner, "Understanding Eye Movements in Reading," Scientific Studies of Reading 1, no. 4 (1997): 317; we do not have comparable estimates for Italian or Spanish, although translations of texts from English to Spanish or Italian in international literacy surveys require an average of 20% more printed space when translating (Steve Dept, cApStAn Linguistic Quality Control, personal communication).
46 Ronald P. Carver, "Reading for One Second, One Minute, or One Year from the Perspective of Rauding Theory," Scientific Studies of Reading 1, no. 1 (1997): 3-43.
47 For example, Richard W. Woodcock, Kevin S. McGrew, and Nancy Mather, Woodcock-Johnson III. Tests of Cognitive Abilities (Itasca, IL: Riverside Publishing, 2001).
48 The native versus nonnative (NATIVESPEAKER) variable is derived based on responses to the background questionnaire. The respondent was considered a native speaker if his or her first language was the assessment language.