Seminar: University Seminar on Cognitive and Behavioral Neuroscience (603)

Date: April 24, 2003

Title: The Mapping of Sound Structure to the Lexicon: Evidence from Normal Subjects and Aphasic Patients

Speaker: Sheila Blumstein, Department of Cognitive Science, Brown University

Attendees: Herb Terrace, Co-Chair, Psychology Department, Columbia University

Robert Remez, Psychology Department, Barnard College

Robert Krauss, Psychology Department, Columbia University

Ezequiel Morsella, Psychology Department, Columbia University

Jennifer Pardo, Psychology Department, Columbia University

Josh Davis, Psychology Department, Columbia University

Jim Magnuson, Psychology Department, Columbia University

Lois Putnam, Psychology Department, Columbia University

Michele Miozzo, Psychology Department, Columbia University

Jont Allen, University of Illinois, Urbana

Gina Cardillo, Psychology Department, Columbia University

John Chen, Columbia University

Kristin Geiger, Columbia University

Jessica Goldberg, Barnard College

Owen Rambow, Columbia University

John Sidtis, New York University

Diana Vanlancker-Sidtis, New York University

Simon Fischer-Blum, Columbia University

Stephen Lowery, Columbia University

Jingtian Wang, NYSPI

Rapporteur: Michael R. Drew

Abstract: 

This research explores how listeners map the properties of sound to the lexicon (the mental dictionary) and investigates the neural basis of such processing. Our experiments with both normal subjects and aphasic patients examined the effects of phonological and acoustic-phonetic structure on lexical processing. Specifically, we investigated the extent to which phonological and acoustic-phonetic modifications of an auditorily presented prime stimulus affect the magnitude of semantic priming to a real word target in a lexical decision task. Results from normal subjects suggest that:

- activation of the lexicon is graded;

- both phonological and acoustic-phonetic structure influence lexical activation;

- the prototypicality of an exemplar member of a phonetic category influences the degree of lexical activation; and

- acoustic-phonetic structure activates not only its lexical representation and lexical network but also the lexical representation and lexical-semantic network of its competitors.

Results from aphasic patients suggest that they have deficits in the dynamics of lexical activation. Broca's aphasics appear to have an overall reduction in lexical activation, whereas Wernicke's aphasics appear to have an increase in lexical activation or a failure to inhibit lexical candidates. The neural systems underlying lexical activation will be considered.

Summary:

Dr. Blumstein’s talk was about the mapping of sound structure to the lexicon, largely focusing on data from normal subjects and aphasic patients. Dr. Blumstein began by remarking that most of what we know about the neural basis of language comes from studies of aphasic patients. These studies led to the “lesion-based model” of language processing, which says that language functions are located primarily in the perisylvian area, the supramarginal gyrus, and Broca’s area. Damage to those areas will produce a language deficit, but the nature of the deficit will vary depending on the location of the lesion. The classical model (Geschwind, 1965) states that damage to anterior language areas (Broca’s area) produces expressive language deficits, including difficulty with articulation and production agrammatism. Damage to the posterior areas produces deficits in language perception and comprehension. Dr. Blumstein’s research indicates that language processing is not so neatly divided between anterior and posterior areas. Rather, damage to frontal areas produces deficits in both language production and lexical processing, indicating that language perception is distributed across both anterior and posterior areas.

Blumstein then described the 5 stages of lexical processing:

1. sound input

2. spectral mapping

3. phonological mapping

4. lexical mapping

5. word selection

The stages of lexical processing can be studied using the semantic priming paradigm (SPP), in which subjects are presented with two phonetic stimuli in succession. The stimuli may be words or nonwords. When the second stimulus (the target) is presented, the subject must indicate whether it was a word or nonword. The measure of interest is the speed with which the choice is made. The task takes advantage of the fact that words activate whole lexical/semantic networks, in addition to their specific referents. When the first stimulus word (the prime) activates a network of which the second (target) word is a member (e.g., CAT—DOG), the lexical decision will be made faster, as compared to trials when the first stimulus is a nonword (e.g., SHLUMP—DOG) or when the two stimuli are unrelated (e.g., NOSE—DOG). The decrease in choice latency seen on trials when the stimuli are semantically related is a semantic priming effect.

In the first series of studies she described, Blumstein examined how phonological changes in the prime words affect performance in the SPP. The experiments address how changes in phonological input affect lexical mapping. The primes could be of three types: a word related to the target (CAT—DOG), a word unrelated to the target (NOSE—DOG), or a nonword that is phonetically similar to a prime (WAT—DOG; GAT—DOG). Results indicated that nonwords can serve as primes, although the priming effect is smaller in magnitude than when real words serve as primes. This finding indicates that nonwords can map onto the lexicon and that activation of the lexicon is graded. The amount of activation depends on the quality of the prime.

In the second series of experiments, Blumstein examined the performance of aphasics on SPP tasks. Both Broca’s (nonfluent) and Wernicke’s (fluent) aphasics show semantic priming with word primes. Their performance diverges when nonword primes are introduced. Broca’s aphasics do not show priming with nonword primes. Wernicke’s aphasics do show priming effects with nonword primes, and, interestingly, the nonword primes are as effective as word primes. The results suggest that Broca’s aphasics have an abnormally stringent criterion for lexical activation, and Wernicke’s aphasics have an abnormally loose criterion for activation.

Dr. Vanlancker-Sidtis: Wouldn’t nonword targets require significantly more time than word targets?

Dr. Blumstein: Yes, but remember that we only look at trials when the target is a word. To prevent response biases, we need to have an equal number of word and nonword targets, but for our analyses we’re only interested in the word target trials.

Dr. Terrace: For the normal words, both Broca’s and Wernicke’s aphasics are about 500ms slower than normals. Why is that?

Dr. Blumstein: Good question. The aphasic patients are just slower.

The next section of the talk focused on spectral mapping. Most of the lexical processing literature ignores spectral mapping, in favor of starting at the higher levels of processing that occur after lexical mapping. Blumstein believes that this constitutes an oversight, not least because, as her earlier studies demonstrate, phonetics can affect lexical processing. It is usually assumed that words prime both semantically (e.g., CAT—DOG) and phonetically (e.g., CAT—MAT) related words. It is also known that phonetic perception is categorical. For instance, the sounds “da” and “ta” differ by only a few milliseconds in the parameter “voice onset time” (VOT). The perceptual boundary between “pa” and “ba” is very sharp, in that only a small change in VOT can completely alter the perceptual experience. Still, there is a range of VOTs for both “pa” and “ba”. Although perceptual processing is categorical, activation of the lexicon is graded and depends on the quality of the prime. In other words, in terms of lexical activation, some category members are better exemplars than others. If subjects are given phonemes of varying quality, their identification performance will reflect sharp category boundaries, but their latency for identifying the stimuli will depend on the quality. Moreover, subjects can make accurate judgments about the quality of the stimuli, even when stimuli of varying quality are judged as being members of the same phonetic catergory. Blumstein’s experiments asked how variations in phoneme quality affect semantic priming. Quality was manipulated by varying VOT. The prime could be a related word (CAT—DOG), an acoustically altered related word (CAT*--DOG, using an extreme VOT for “cat”), or an unrelated word (NOSE—DOG). If lexical activation is graded, then the low quality stimulus (CAT*) will prime DOG, but the magnitude of the priming effect will be smaller than that for the high quality stimulus. Importantly, both the high and low quality stimuli were consistently identified correctly. In one experiment there was 50ms interval between prime and target, and in a second experiment there was a 250ms interval. In the 50ms experiment there was a significant effect of acoustic quality on priming, such that better quality stimuli produced bigger priming effects. At the 250ms interstimulus interval there was no effect of quality on priming. The results suggest that the acoustic quality of the stimulus affects lexical activation, but effect is brief (less than 250ms). Blumstein also examined the effects of acoustic competitors. In some instances, the acoustically altered stimuli were expected to activate a phonetically related word, a lexical competitor (e.g., TIME* activates DIME), and in other instance they were not (e.g., CAT* would activate a nonword, GAT, which is not a lexical competitor). Blumstein found that lexical competition produced an overall slowing in lexical decisions but did not moderate lexical priming.

Blumstein also examined the performance of aphasics on this task. Broca’s aphasics perform like normals in the 50ms condition: they show significant priming and there is a significant reduction in priming for low-quality stimuli. The effect persists even at the 250ms interstimulus interval. In the presence of a lexical competitor, however, Broca’s aphasics no longer show semantic priming for low-quality stimuli. Another experiment more explicitly showed that acoustically modified words prime their lexical competitors (e.g., TIME* primes DIME, which primes PENNY) in normals. This effect persisted over the 250ms interval. Broca’s aphasics, however, did not show semantic priming of lexical competitors with the acoustically degraded stimuli. The experiments illustrate the following:

1. the lexical network is a highly interactive system, involving both the expressive and receptive elements of the classical system.

2. there is graded activation at multiple levels of the network.

3. low level acoustic information affects lexical semantics

In the next series of experiments, Blumstein more directly examined the neural substrates of lexical processing using fMRI. There has been a great deal of imaging work on lexical semantic processing, and this work has shown both anterior and posterior activation duration lexical decision tasks. Nevertheless, the work has focused primarily on frontal activation. Some researchers have gone so far as to claim that lexical semantics is a frontal-mediated process. Many of these studies used poorly designed behavioral tasks. For instance, one study asked subjects to provide a related verb for each noun presented (e.g., CAT—MEOW). Blumstein hypothesized that frontal activation would be minimal in the SPP task because the semantics are implied and no overt semantic decision is required.

Dr. Remez: Does anyone have a model of the processes that intervene between contact –e.g., “I heard a word”— and the report of “yes.”

Dr. Blumstein: I don’t think so.

The SPP task was as follows. All primes were words, either semantically related or unrelated to the target. Half the targets were words and half were nonwords. Subjects were required to indicate whether the targets were words or nonwords. All subjects showed semantic priming, in that related targets decreased choice latency. Unrelated, related, and nonword primes all activated the same neural systems (relative to a resting baseline): largely the left posterior and frontal language areas. The unrelated primes produced more activation than did the related primes. (Activation was assessed during presentation of the target word.) Also, word primes produced more activation than nonwords in the middle temporal gyrus, the cuneus, the angular gyrus, and the left anterior cingulate. Blumstein also showed a graph of the time-course of activation that indicated greater activation for unrelated than for related words. A participant pointed out, however, that the difference may reflect a shifting baseline and should be interpreted with caution. The pattern of greater activation for unrelated than related words is consistent with repetition priming data indicating that the magnitude of activation decreases with successive presentations of the same word. Blumstein’s fMRI data suggest that posterior (temporal parietal) brain structures are the areas where lexical semantic representations are stored. There was no significant activation of the inferior frontal gyrus, which was implicated in earlier studies of lexical processing. This means that the inferior frontal gyrus may only be activated in tasks where there is an overt semantic decision. The inferior frontal gyrus may mediate semantic working memory or other semantic executive functions. Blumstein hypothesized that the inferior frontal gyrus would be activated if subjects were required to explicitly compare the prime and target, such as if they were asked to indicate whether the two are related.

Discussion:

Dr. Magnuson: Have you tried to think of tasks other than the SPP that would help you get away from the frontal demands, because the SPP task still requires the subject to make a lexical decision?

Dr. Blumstein: I’m so new in functional neuroimaging, I haven’t really tried. It is going to be hard to find a task that is not metalinguistic. I think it is a good idea not to just do lexical decision tasks. Some people actually have looked at phoneme monitoring, but that is also metalinguistic.

Dr. Magnuson: How about eye-tracking?

Dr. Blumstein: There is a graduate student in our department doing that with aphasics right now. Yes, that is a great idea; a much more natural task.

Dr. Miozzo: It is intriguing that you did not find areas that are more activated for nonwords, whereas if you look at the reaction times, it takes longer to respond to nonwords. Is it that it takes longer to respond to nonwords because there is overall less activation?

Dr. Blumstein: Sure. And a nice feature of the data is that although nonwords take longer to respond to, there is no increase in neural activation. In many paradigms, the amount of neural activation tends to be correlated with the difficulty of the task.

Dr. Allen: Has anyone looked at the confusion matrices of the nonsense CV’s?

Dr. Blumstein: Not that I know of.

Dr. Allen : That’s basically where I thought you were going. Why not just look at phone errors as a function of your VOT distinction, and also add noise and damage the words that way. This would get around the whole reaction time paradigm, which is hard to model in a quantitative way.

Dr. Blumstein: The problem is that the aphasics make many production errors, and so we don’t want them to have to speak.

Dr. Allen: You could use touch screens or a mouse.

Dr. Blumstein: Well, we’d have to make sure they can read. A lot of aphasics have trouble with reading. We have looked at whether noise would affect the magnitude of priming, and it does not. We have also looked at speaker, and that also did not affect the magnitude of priming.

Dr. Morsella: I have two questions. First, is semantic priming due to feature overlap? Two, what if you presented a bad “T,” but also presented visual information suggesting a “D,” as in the McGurk effect? Would this prime both “T” and “D” or would this only prime “D,” because the perceptual experience would be of “D”?

Dr. Blumstein: As for the first question, I don’t have a clue; we’re not making any claims in that regard. As for the second question…

Dr. Remez: You can actually get priming of PENNY with TIME even if it is the best example of TIME. By moving the phone to a slightly imperfect example, you are just increasing the ambiguity and pushing the priming of PENNY higher.

Dr. Blumstein: We do not get priming with TIME—PENNY. In theory we should get it, but perhaps we did not have enough power to detect the effect.

Gina Cardillo: I noticed that in your behavioral task, the primes could be either words or nonwords, but in the fMRI task the primes were all words. Is there an asymmetry in activation with respect to nonwords priming words versus words priming nonwords? In other words, why were there no nonword primes in the fMRI study?

Dr. Blumstein: We have done that. We did not see any inhibition by nonword primes. If you vary the proportion of related words, you see that Broca’s aphasics are very much driven by strategic processes. So if they know that the outcome is predominantly one or the other, their behavior is very sensitive to that.