Test Design

Each task used in the literacy assessments represents a piece of evidence about a person's literacy. While the goal of the assessments is to develop the best possible picture of an individual's skills and abilities, the tests cannot include endless tasks. Key central features were considered during the development of the tests.

We identified three task characteristics — adult contexts/content, materials/texts, and processes/strategies — and used them in the construction of tasks.

Adult Contexts/Content

Adults do not read written materials in a vacuum. Rather, they read these materials within a particular context, or for a particular purpose. We selected materials for the assessment that represent a variety of contexts and contents. This is to ensure that no one group of adults is either advantaged or disadvantaged due to the content included in the assessment.

Six adult context/content categories have been identified:

  • Home and family: Materials cover interpersonal relationships, personal finance, housing and insurance
  • Health and safety: Materials cover drugs and alcohol, disease prevention and treatment, safety and accident prevention, first aid, emergencies and staying healthy
  • Community and citizenship: Materials cover staying informed and community resources
  • Consumer economics: Materials cover credit and banking, savings, advertising, making purchases and maintaining personal possessions
  • Work: Materials cover various occupations but not job-specific texts, finding employment, finance and being on the job
  • Leisure and recreation: Materials cover travel, recreational activities and restaurants

Materials/Texts

What is critical to the design and interpretation of the scores is the range of the text material. A key distinction among texts is their classification into continuous and noncontinuous texts.

Continuous texts are classified as prose and are sentences organized into paragraphs. Organization occurs by paragraph setting, indentation and the breakdown of text by headings that help the reader to recognize the organization of the text. The primary classification of continuous texts is by rhetorical purpose or text type. These include expository, descriptive, argumentative and injunctive.

Noncontinuous texts are organized differently and allow the reader to employ different strategies for entering and extracting information. On the surface, these texts appear to have many different organizational patterns or formats, ranging from tables and schedules, to charts and graphs, and from maps to forms.

However, the organizational patterns for these types of texts (what literacy researchers P.B. Mosenthal and I.S. Kirsch called documents in the Journal of Reading in their monthly column series, which ran from October 1989 to May 1991) have one of four basic structures:

  • simple list
  • combined list
  • intersected list
  • nested list

Together, these four document types make up matrix documents, or noncontinuous texts, with clearly defined rows and columns. They are also closely related to other noncontinuous texts that these authors refer to as graphic, locative and entry documents.

The distinction between continuous and noncontinuous texts formed the basis for two of the three literacy scales used in these adult literacy surveys. Continuous texts were the basis for tasks along the prose scale, while noncontinuous texts formed the basis along the document scale. The quantitative scale included texts that were both continuous and noncontinuous. The distinguishing characteristic for this scale was that respondents needed to identify and perform one or more arithmetic operations based on information in the texts.

Processes/Strategies

This task characteristic refers to the way examinees process text to respond to a question. It includes the processes used to relate information in the question to the information in the text, as well as the processes needed to identify or construct the correct response.

Locate tasks require examinees to match one or more features of information in the question to either identical or similar information in the text.

Cycle tasks also require examinees to match one or more features of information, but unlike locating tasks, they require respondents to engage in a series of feature matches to satisfy conditions stated in the question.

Integrate tasks require examinees to pull together two or more pieces of information from the text according to some type of specified relation. For example, the examinees may need to identify similarities, differences, degree or cause-and-effect relations. The needed information may be located within a single paragraph, or it may appear in different paragraphs or sections of the text. In integrating information, examinees draw upon information categories provided in a question to locate the corresponding information in the text. They then relate the text information associated with these different categories based upon the relation term specified in the question.

Generate tasks require respondents to process the information located in various parts of a text, and create a new category or main idea or summarization by drawing on their knowledge about a subject or by making a broad text-based inference.

Formulate and calculate tasks require respondents to apply one or more basic arithmetic operations. Before performing these calculations, respondents must locate the numbers in a text and determine which operation(s) are needed to respond correctly.

A number of variables were found to influence the difficulty of the literacy tasks. The type of process involved in locating, cycling, integrating and generating information is one important factor influencing task difficulty. The complexity of tasks associated with these processes/strategies can be influenced by several other variables, such as:

  • the number of categories or features of information the reader is required to process
  • the extent to which information given in the question or directive is obviously related to the information contained in the text
  • the length and complexity of the text itself

Task complexity is also influenced by factors such as the amount of textual information and its location — and whether or not these factors fully answer the question at hand. On the quantitative scale, the number and type of arithmetic operation are important variables in influencing the overall difficulty of a task.

See also: