Uses and Misuses of Processing Tests

If you have ever been involved in an educational evaluation for learning disabilities — whether as a parent, teacher, or in some other role — you have probably heard about “processing tests” and “processing disorders.” Many aspects of these evaluations can be problematic, but perhaps none more so than those entailing the use (and potential misuse) of processing measures. Yet the right measures, carefully interpreted, can be enormously helpful in education. In this article, I will discuss the benefits of processing measures, as well as their limitations. Although I will focus primarily on the area of reading, the possible uses and misuses of processing measures are similar across other domains of learning disabilities, such as mathematics and written expression.

How processing measures are used in education

The Individuals with Disabilities Education Act, or IDEA, defines learning disabilities as involving “a disorder in one or more of the basic psychological processes.” Many state regulations also include the concept of a processing disorder in their definitions of learning disabilities. In education, determining the presence of a processing disorder generally involves using tests of memory, language, auditory processing, or visual processing–abilities believed to underlie academic learning–rather than tests with actual academic content. Although educators often employ the latter types of tests to document an IQ-achievement discrepancy or to evaluate progress, they usually do not view these tests as measures of processing.

However, cognitive psychologists have conceptualized processing measures more broadly than have many educators. For example, cognitive psychologists interested in reading disability (or dyslexia) have emphasized that a core deficit for dyslexics appears to involve the use of phonological processes in reading (Rack, Snowling, & Olson, 1992). A measure widely used by scientific investigators to assess these abilities is the Word Attack subtest of the Woodcock Reading Mastery Tests, which requires the individual to read out-of-context pseudowords (nonsense words) such as taff and grum. In contrast, educators would more commonly use this subtest to document low reading achievement than as evidence of a “processing disorder.”

Here I will use the term “processing” broadly to include tests with, as well as without, obvious academic content.

How processing tests can be helpful

Processing measures can be quite helpful in early identification of reading difficulties and in planning reading instruction. Trying to teach a child to read without information about the underlying cognitive processes important in reading is like trying to figure out how your car works without being able to look under the hood. An essential point, however, is that the measures used have to be the right ones. Just as looking in the trunk probably won’t tell you much about whether your car engine is working properly–and if not, what the problem is–using the wrong processing measures won’t provide much insight into children’s (or adults’) reading difficulties. For poor readers, the most relevant processing tests tend to be measures directly related to reading (e.g., letter identification or out-of-context word decoding) or measures of linguistic processes (e.g., phonological awareness). Also, the tests used in any educational evaluation must be technically adequate; for instance, they must have adequate norms, reliability, and validity. (Potential sources of information about the technical adequacy of specific tests can be found at the end of this article.)

Of course, the processing measures most relevant to mathematics or written expression are not identical to those most useful in reading. Thus, in choosing processing measures, an examiner should consider the academic domain(s) in which a child experiences difficulty. Unfortunately, in education, processing measures are not necessarily selected on this basis. Rather, it is quite common for all children at a particular age level to be given the same, usually fairly extensive, battery of processing measures; a low score on any measure then may be assumed to reflect a “processing disorder.” One problem with this scattershot approach, in addition to the fact that it may not yield educationally useful information, is that it capitalizes on chance. In other words, if enough tests are given to a person, even to someone with no learning problems at all, he or she probably will eventually get a low score on at least one test. This possibility is especially likely in educational evaluations for learning disabilities because many of the tests used in these evaluations lack technical adequacy (Moats & Lyon, 1993). In this situation, an individual’s low score on a processing measure may be completely meaningless.

I. Early identification of reading difficulties

Much of the research on early identification of reading difficulties has focused on using measures of various cognitive abilities to predict future reading achievement in preschool or kindergarten children who are nonreaders. This work suggests that the following three abilities strongly predict beginning reading achievement: letter identification, phonological awareness (or sensitivity to the sounds in spoken words), and the capacity to rapidly name arrays of letters, digits, colors, or pictured objects (Wagner & Torgesen, 1987; Wolf, 1991). In addition, in a very thorough review of the literature on early identification, Scarborough (1998) found that expressive vocabulary, sometimes called confrontation naming, is highly predictive of early reading achievement. Expressive vocabulary is usually assessed by asking children to name individual drawings or pictures of objects. Finally, general language ability also predicts reading achievement, and children with broad language difficulties or delays clearly are at risk for reading problems.

Although these reading-related cognitive measures are quite useful in predicting reading achievement for large groups of youngsters, they fall far short of perfection in individual screening. For instance, on average in the studies reviewed by Scarborough (1998), even screening batteries involving combinations of tests failed to identify over 20% of kindergartners who would later go on to experience reading difficulties; and they erroneously identified an even larger percentage of kindergartners as “at risk” who would turn out not to have reading problems at all. Nevertheless, if their shortcomings are recognized, these measures can still be useful in educational testing. And as Scarborough points out, routine screening using these tests might well be worthwhile for young children already known to be at risk for reading difficulties, such as those with oral language impairments or a family history of reading problems.

At least one other type of measure can be helpful in early identification of somewhat older children who have experienced formal reading instruction. This measure involves out-of-context word decoding, as on the Word Attack subtest of the Woodcock Reading Mastery Tests, mentioned earlier. The use of pseudowords, such as glift, is important to provide a measure of pure decoding as opposed to sight-word recognition. It appears that children with significant weaknesses in out-of-context word decoding are at risk for future reading problems, even when they have adequate sight-word knowledge and can read better in context than in isolation (e.g., Byrne, Freebody, & Gates, 1992). The problem for these youngsters is that, as they advance in school and the texts that they have to read become harder, it becomes increasingly difficult to compensate for poor word decoding by using sight-word or contextual abilities.

II. Instructional planning in reading

One way that processing measures can be useful in instructional planning is in pinpointing weak areas in need of direct instruction. For instance, there is abundant evidence that training phonological awareness, especially in conjunction with a program of formal reading instruction, can improve reading achievement in many at-risk youngsters (e.g., Hatcher, Hulme, & Ellis, 1994). Similarly, cognitive strategy instruction, especially when integrated with content instruction, may benefit reading comprehension in older children (Ellis, 1993). And intensive, systematic instruction in word decoding improves decoding accuracy both in young at-risk youngsters and in older poor readers (Torgesen, 1998). (However, as discussed further below, it should not be assumed that training of all weak areas is warranted.)

A second way that processing measures may be useful in educational planning is in giving information about an individual’s overall cognitive profile in reading, which in turn may yield implications for instruction. For example, a youngster who obtains low scores on reading-comprehension tests, but who has an above-average verbal IQ and good listening comprehension, is not likely to have a true comprehension problem. Rather, it is more likely that this kind of youngster has problems revolving around word decoding that is inaccurate, nonautomatic, or both. In this case, in order to improve reading comprehension, the educational program must build accuracy and fluency of word decoding.

Table 1 summarizes some of the cognitive processes discussed so far, along with examples of tasks and individually-administered standardized tests that often are used in evaluating each type of ability.

The limitations of processing measures

Although the right processing measures can be very valuable in early identification of reading problems and in educational planning, these measures also have some important limitations, such as the following:

I. Processing measures cannot be used to determine the ultimate causation of an individual’s learning problems

In educational evaluations for learning disabilities, poor performance on processing measures frequently is assumed to result entirely from an intrinsic deficit or disability. However, processing tests are at the level of psychological description; they cannot be used to determine whether an individual’s learning problems are primarily the result of “nature” or “nurture.” For example, although phonological-processing abilities such as phonological awareness appear to be strongly genetically influenced, they also are shaped by environmental factors (Mann, 1994; Olson, Rack, Conners, DeFries, & Fulker, 1991), such as home literacy experiences and the nature of classroom reading instruction. Behavioral geneticists interested in reading disability have emphasized that genetic influences never develop in isolation, but always in interaction with the environment, both the surrounding biological environment (i.e., the individual child’s unique biological make-up) and day-to-day experiences. Moreover, particularly in older children and adults, some processing weaknesses actually may be caused by poor reading itself. For example, a lack of exposure to text–because of failure to engage in independent reading–may lead to problems with orthographic-processing tasks, such as those requiring spelling knowledge or reading of phonetically irregular words (Stanovich, Siegel, Gottardo, Chiappe, & Sidhu, 1997).

Current federal and state educational guidelines charge learning-disabilities specialists with sorting genuine cases of learning disabilities from other types of achievement problems. Unfortunately, however, at present it is impossible for special educators to meet this charge, at least for most cases of poor reading. Although there is a large research literature on the biological underpinnings of reading disability, most measures that have been employed in this research are not feasible to use in educational screening, nor could they yet be used to separate genuine cases of reading disability from other types of reading failure. And processing tests cannot be used to identify genuine cases of reading disability, because no unique processing profile has been associated with true reading disability, and because poor performance on processing tests is not proof of an intrinsic disorder in learning.

II. Interpretation of processing measures can be difficult.

Tests that appear to measure primarily one set of processes may actually tap very different processes. For example, most of the visual-processing measures used in education involve paper-and-pencil tests or relatively long exposures of test items. These measures often tap linguistic, memory, or motor abilities instead of–or in addition to–visual processing (Vellutino, 1979). These problems make it impossible to know whether children who obtain low scores on educational tests of visual processing actually have a visual-processing weakness, or some other type of weakness. For these reasons, scientists interested in the role of visual processes in reading have developed purer measures of visual processing, often involving tachistoscopic presentation of test items. However, these kinds of measures are rarely, if ever, used in education.

Likewise, Scarborough (1998) suggests that tests of expressive vocabulary may predict beginning reading achievement because they tap phonological processes important in early reading–not necessarily because expressive vocabulary itself is essential to early reading.

Even processing measures with academic content, whose interpretation may appear to be straightforward, do not always measure exactly what they are believed to measure. For example, Nation and Snowling (1997) studied two different tests of reading comprehension, a measure that used a sentence completion or cloze format (in which the child has to read a sentence and fill in a blank with a word that makes sense, e.g., “The fish were swimming in the ______”) and a different measure that required children to read stories aloud and answer questions about them. They found that the former measure, using the sentence-completion format, related more strongly to word recognition than to broad language comprehension; only the measure involving reading stories and answering questions about them related strongly to language comprehension.

In other words, although both types of tests were labeled as measures of “reading comprehension,” they were actually tapping different underlying processes–because of differences in the tasks they used. In fact, Nation and Snowling found some children with significant language-comprehension weaknesses who scored within the average range on the sentence-completion test. Conversely, they point out that a dyslexic youngster might get a low score on the sentence-completion test and be erroneously assumed to have a reading-comprehension problem, when his or her problems really center upon poor word decoding.

III. Considerable caution is required in using information from processing measures in instructional planning.

One of the reasons that there has been so much enthusiasm in the scientific community about phonological awareness is that, not only has phonological awareness been shown to predict reading achievement, but training it has actually proved beneficial to reading. As mentioned earlier, strategic abilities in reading also appear to be trainable. Unfortunately, however, we still know little about how, or even whether, to train many reading-related cognitive processes.

For example, although rapid naming tasks predict early reading achievement, currently there is little evidence that directly training those tasks improves reading. Most scientific investigators of reading are quite guarded about recommending training of cognitive processes to educators. In order to make training worthwhile, educators need to know that the processes in question have a causal relationship to reading and are trainable, as well as that training them improves reading achievement. This knowledge is not yet available for many of the cognitive processes that have been studied in reading.

Finally, information from processing measures sometimes is used to match a method of reading instruction to a child’s underlying “modality preference” or “reading style.” For instance, a youngster with visual-processing strengths and auditory-processing weaknesses might be prescribed a sight-word or holistic reading method, whereas a youngster with the opposite profile (i.e., strong auditory, weak visual) might be prescribed phonics. Research literature reviews have repeatedly concluded that matching reading method to modality preference is not effective (Arter & Jenkins, 1977; Kavale, Hirshoren, & Forness, 1998). Furthermore, it should be noted that many poor readers appear to have auditory-processing weaknesses on educational tests–because these tests often use linguistic stimuli such as words or numbers, and poor readers frequently have phonological-processing problems that influence their performance on linguistic tasks. (In other words, what may be interpreted as a global auditory-processing deficit may really be a phonological-processing deficit.) If these youngsters are prescribed a pure sight-word or holistic method instead of explicit teaching of word decoding (i.e., phonics), their problems may be compounded.

IV. There is much more to achievement than what is captured by conventional processing measures.

Processing measures do not tap, or only indirectly measure, many individual differences that ultimately may be quite important to achievement. For instance, educational researchers (e.g., Baumann & Duffy, 1997; Farkas, 1996) have found that noncognitive variables such as persistence, work habits, interests, and motivation, play an important role in reading achievement for children from a range of backgrounds and ability levels. Moreover, noncognitive variables may influence the development of cognitive abilities over time, because individuals who are motivated to persist at certain tasks–such as reading difficult books–are more likely to improve the cognitive abilities needed for those tasks. Similarly, cognitive abilities such as creativity and practical intelligence (e.g., Sternberg, 1996) usually are not tapped by traditional educational measures, but may strongly influence an individual’s success in life. Thus, even if a youngster has a significant and enduring deficit in some processing ability–and there is evidence to indicate that many individuals with reading disability may have this kind of long-term weakness in phonological processing–this deficit does not necessarily preclude going to college, pursuing a profession, or having a satisfying, successful life.

Of course, teachers and parents should still be concerned about signs of reading difficulties and should do all they can to address these problems. However, it also is helpful to keep a sense of perspective about reading difficulties–and to try to communicate this perspective to youngsters who are struggling in school.

How processing measures should be used in educational evaluations

To sum up, here are some characteristics to look for in educational evaluations involving processing measures:

The measures should tap important component processes in the individual’s domain(s) of difficulty. In reading, the cognitive processes that are most essential to consider may vary somewhat with age and level of reading achievement. For young and beginning readers, important component processes include phonological awareness, letter identification, listening comprehension, and accuracy of decoding single words. For older and more advanced readers, it may be especially important to consider speed and fluency of reading, the use of comprehension strategies, and detailed analyses of reading comprehension (e.g., literal vs. inferential comprehension, or comprehension of narratives vs. expository text).

Processing measures should be technically adequate. Information on the technical adequacy of many commercial tests can be found in sources such as Buros’ mental measurements yearbooks, Sweetland and Keyser (1993), basic textbooks on special-education assessment (e.g., Salvia & Ysseldyke, 1995), and in the test manual itself.

Processing measures should not be the basis for assumptions about the ultimate causation of an individual’s difficulties. Instead, they should be viewed as a source of educationally useful information about which children may be at particular risk for reading problems and about an individual’s current pattern of strengths and weaknesses in various reading-related cognitive processes.

Information from processing tests should make sense. This information should be consistent with, but should also complement and clarify, observations of the individual’s everyday performance in reading. If test results cannot be reconciled with impartial, well-informed observations of day-to-day performance, the accuracy of test results should be questioned.

This information also should lead to specific, educationally useful recommendations that are tailored to an individual’s needs. Very broad recommendations, such as “needs structure” or “needs to learn organizational skills” apply to almost all low achievers (and to some high achievers as well). More useful are recommendations such as “needs to learn strategies for decoding multisyllable words,” or “needs to build speed and fluency of reading in context,” along with specific techniques or programs for meeting these goals.

Certain pitfalls in using processing measures in instructional programming should be avoided. These pitfalls include training of cognitive processes in total isolation from academic instruction, attempted training of processes that have not been shown to be trainable, and matching reading method to modality preference.

8/3/98

About the author: Louise Spear-Swerling is a professor of special education at Southern Connecticut State University. She has taught both special education and regular classes in public schools and now teaches graduate and undergraduate courses in learning disabilities. She is also the co-author of Off Track: When Poor Readers Become ‘Learning Disabled’(Westview Press, ©1996, ISBN 0-8133-8756-6).