Multisensory Perception

We have largely discussed sensory perception in a Cartesian way, as though each of the senses were an independent function. Although we see orderly ways in which these things are organized, and we can relate that to the known kinds of developmental processes, the emergent nature of perception in each type of sense is still rather elusive. In fact, although many of these sensory systems do function without input from others, higher sensory processing often involves their interaction.

Morphological, visual, olfactory, auditory information is routinely integrated in evaluating threat postures, prey or predator behavior, and the like. Perception of smell and taste interact to produce a more complex sensation, and spoken language comprehension can be more precise when complemented with visual processing of lip movements. Significant interaction takes place between visual and auditory input in vertebrate development, and visual input can influence tonotopic mapping in the auditory cortex (King 1999). Indeed, it is this intersystem integration that constitutes behavioral responses to the environment, a fact reflected at the neural level as well. These statements are true of invertebrates as well as vertebrates and indeed, in their way, of plants and single-celled organisms, too.

In many vertebrates, some processing actually takes place in multisensory neurons. Cortical and subcortical afferents from a variety of different modalities converge in these neurons and allow enhanced (or sometimes depressed) perception of an event. That is, neurons that respond to visual cues may also receive audi tory and touch cues and then issue multiple motor commands, controlling the orientation of the head, eyes, ears, and so forth, in response.

Located in different regions of the neuraxis, the most well-understood being the superior colliculus of the midbrain, where sensory cues are received and motor commands initiated, these neurons have overlapping receptive fields for each separate modality from which they receive signal (e.g., Kadunce et al. 2001; Meredith and Stein 1986; Stein et al. 2001). Interestingly, although they already have multiple receptive fields and act as fully developed multisensory neurons in their signal reception and sending capacities, these neurons are not able to process multiple modalities at birth. Their multisensory processing capabilities only develop with increasing sensory experience (Wallace and Stein 2001).

This fact is not a surprise, however, as it is known that if one sensory modality is lost, the parts of the brain that perceived and controlled responses to that sense can be recruited by other sensory modalities. This rewiring or remapping does not need to happen only during development; the brain is plastic throughout life. The visual cortex of someone who loses vision as an adult, for example, can be partially recruited for touch or hearing. As with many aspects of perception, sensory and mul-tisensory processing in the brain is the product of hard-wiring and experience.


organismal responses

It is difficult for us to assess the relationship between detection and perception without involving our own experience and notions that connect awareness and consciousness. Indeed, that experience places limits on our attempts to understand perception generally, especially in other species. Thus, a discussion of consciousness itself, speculative though it must necessarily be, is in order.

Nerve nets and ganglia allow some forms of coordinated or integrated responses. Plants, however, do not have any kind of organized nervous system, and, as we have seen, they manage to have coordinated and integrated responses to environmental signals nonetheless; therefore, a centralized perception of the environment or a higher-level organization of response is not necessary for organized multicellular life. With respect to plants and the simplest nerve nets, we may say that "all" this is is reflex triggering of neural impulses by input impulses—a purely mechanical process. In the case of ganglia and more complex systems, we might say that this is "simply" a more highly organized form of the same thing. Cognition—the real thing?—involves similar chemical reactions. What is the difference, and, especially, what is the difference when it comes to consciousness?

One standard if nearly metaphysical answer is that these higher-order processes are analogous to electromagnetic fields in physics: when many neurons fire at once, a higher-order emergent phenomenon occurs, say a "field" of "electro-chemo-magnetic" energy, and if enough neurons fire at once, that becomes the phenomenon that we, at least, experience as consciousness. As humans, we tend to equate this with the organized unifying phenomenon we refer to as perception.

How this would work or what it means in scientific rather than metaphysical terms are difficult questions. But let us take the ideas as generically correct; if perception is an emergent phenomenon, can a simple organism with a neural net, or even a ganglion itself, also generate some form of emergent "aura" of awareness that we would recognize relevant to our own human experiences? For that matter, because plants also organize reactions in a complex way, is there any sense in which we could refer to this as plant "perception"?

We have several times referred to the complex social organization and problem solving abilities of ants and other social insects, who have the tiniest of brains. We can list neural wiring patterns, name some lumps and bumps and connection zones in the brain, and identify neurotransmitters, ion channel genes, and adhesion and signaling factors and receptors that are expressed in brain development. But do we understand what this means any better now than Aristotle did 2,400 years ago?

Natura non Facit Saltum: Perception is Not an All-or-Nothing Phenomenon

Humans with severe forms of epilepsy have sometimes been treated by severing the corpus callosum, one of the major neural throughways that connect the left and right sides of the brain. In these "split-brain" individuals, who behave and report still feeling their normal identity, the evidence suggests that only one hemisphere of the brain (usually, the left) is the seat of consciousness. The other hemisphere is a fully functional, problem-solving entity that can even be shown to be self-aware. But it may not have consciousness (an important part of this may be aspects of language and verbal expression, usually controlled by the left side). Perception and sophisticated problem solving are possible without explicit consciousness.

It is clear that there is no one such thing as consciousness or the experience of awareness. We sleep, dream, and can drive cars without "thinking" about it. Each person experiences things in different ways at different times and differently from other people. Some people have a "feel" for science, music, personal interactions, or the flow of a hockey game. Some smell or see things others completely miss, facts that in this case we know directly, from studies of opsin spectral analysis or odorant-specific sniff-testing. Some people even claim to have direct contact with the immaterial world, whereas others who lack that experience strongly declare that to be delusional. Fanciful as such contact experience may seem, areas of the brain that are responsible (for the experience, whether or not the reality) are identified by studies of brain activity scans and the association of some such activities with epileptic episodes and so on.

The most central of all observations of evolution is that systems in related organisms are similar because of shared descent with modification. There is considerable freedom in evolution, but not complete freedom; thus, this kind of relationship has an organic reality. There is no reason whatever not to extend the same principles to perception and consciousness itself.

Darwin was committed to the notion natura non facit saltum—nature does not take leaps. To him, this principle gave evolution by natural selection its plausibility, and he used this principle in defending many attacks on his theory (especially in expanded discussion in the sixth edition of Origin of Species).His inference was that evolution and selection worked gradually over time. Traits did not emerge de novo. This in a sense is the basis of the modern view by which the step-by-step evolution of complex traits occurs.

Evolutionary biologists have come, perhaps somewhat reluctantly, to accept "punctuated" events in evolution in two senses. First, it is generally accepted that there can be times of acceleration in the rate of change relative to longer times of slower change. This can be brought about by things like the invasion by a species of a new territory or by rapid climate change (or by intense use of antibiotics). Secondly, homeotic change is recognized as a mode of evolutionary "jumps" in which the number of segments like digits or vertebrae can change as a result of changes in meristic patterning processes. In a sense, the latter is just a quantitative change, requiring no sudden novel mechanisms. But beyond that, we have no good examples to persuade us that a new trait can suddenly appear.

It is thus interesting that a darwinian (gradualistic) view of consciousness should be controversial, but it seems to be so. Many scientists are reluctant even to acknowledge that chimpanzees share the human experience of consciousness in more than a rudimentary way. In what can be characterized as at least a bit anthropocentric, since humans and chimpanzees have evolved numerous important differences in other traits, efforts to explain our evolutionary difference from apes has often been focused on the brain. The classic example is the famous but vain struggle by Richard Owen in the 1800s to find a trait unique to the human brain. He thought he had done so with the hippocampus major but was famously embarrassed in that notion by the ever-combative Thomas Huxley. Recent invocations of the brain-centered bias have demonstrated genetic differences between humans and chimpanzees that involve language or differential gene expression in the brain (Enard et al. 2002a; Enard et al. 2002b).

Any application of human experience to "lesser" species—even dogs and their emotions, not to mention the social behavior of ants—is typically denigrated as anthropomorphizing (or worse). However, some authors have tried to justify the position of humans as part of the natural world (rather than above it, by virtue of our unique traits, like consciousness) by insisting that consciousness has arisen out of a phenomenon that somehow pertains to everything. These views have typically come out of philosophical or religious perspectives under notions that might generally be referred to as animism. Animism asserts some type of universal internal awareness and drive, of which the human mind is the logical, inevitable, or creational acme. Famous biologists and writers on biology including Lamarck, Henri Bergson, and Teilhard de Chardin (to name some of the most prominent) have held such views, often including rudiments in atoms themselves to make the system universal (e.g., Chardin's Phenomenon of Man). Even one of the cofounders of population genetics theory, J. B. S. Haldane, made this kind of point (Haldane 1932).

We can look at this subject from an entirely rationalistic point of view, with no mysticism attached. All we need do is apply the same evolutionary notions that we apply to other traits, asserting the gradual or quasi-gradual origins of new traits (no saltations). Natura non facit saltum. It follows almost automatically that other species would at the very least have identifiable run-ups to what we experience as consciousness, and these should involve the same sensory neural-integrating processes in other species that they do in humans. We may not know what context-dependent, intermittently flickering, or partial consciousness feels like. Nor are there unambiguous criteria for speculating how far into the range of species this phenomenon may reach. But by any consistent standard of evolution, this important trait must have evolved over time and rudiments of some sort may still characterize much of animal life.

It is hard to accept that even insects might have complex perceptions to go with the behavior that has impressed so many. Henry W. Bates, one-time exploring companion with Wallace, remarked of Amazonian sand wasps: "The action of the wasp [in building a nest for its young and stocking it with paralyzed insects] would be said to be instinctive; but it seems plain that the instinct is no mysterious and unintelligible agent, but a mental process in each individual, differing from the same in man only by its unerring certainty" (Bates 1863). Anyone watching social insects can easily see that their behavior is to some extent open-ended in the sense we have used the term in this book: they may or may not have a fixed or limited repertoire of basic interactions, but they use them in nontrivial, context-dependent ways that we should not diminish by a facile assumption that what appears so complex and well organized must be "just" instinct instead. One day we may develop some way to understand what the experience is like to be an ant or wasp.

The dramatic difference seen today between humans and others seems to rest clearly on language and/or whatever symboling abilities that entails and entailed evolutionarily. If we compare electrical engineers or Shakespeare, with chimpanzees, the difference of course seems qualitative. However, several species of human-like ancestors have existed in the past, and we can trace through the fossils the progressive increase in cortical brain size. It seems hard to envision that this was not accompanied by progressive increase in "consciousness." Not even Homo erectus suddenly emerged with consciousness.

To assert that consciousness is a totally new phenomenon or even represented solely by ourselves, would verge uncomfortably on an essentially unprecedented view of the sudden appearance of what Richard Goldschmidt, in opposition to the Darwinian assertion of graduation, famously termed a "hopeful monster" (a baby who could speak language to parents who couldn't?). It would be like invoking spontaneous generation or a form of special creationism—the very last thing most biologists would want to be accused of.


It is easy to say that perception is an organism's way to develop an internal map of the external world, but, given that different species thrive with the external world internalized in such diverse ways and that even individuals within a species do not completely share internal maps, no specific internal map can be said to be a "true" representation of the outside world, but only one that suffices for the particular individual or species. In that sense, the subset of cues an animal takes from the environment are what its ancestors required to live and reproduce; however, in a sort of darwinian feedback mechanism, these cues also drive the evolution of the pathways that allow the animal to collect the information that it needs. And, in an echo of Kant, an organism's world only is those aspects it can perceive.

There are many other fundamental questions about sensory perception that we don't yet know how to answer. For example, given that there are critical periods for synapse formation, and if external stimuli and experience are crucial to their formation, how can stereotyped patterns arise? How much of what appears to be pattern merely reflects the shared experience (e.g., uterine environment) of all members of a species? If there are stereotyped patterns, brain areas "for" different senses, how can these areas also be so plastic, so readily recruited by other senses, in the event of, say, a brain lesion or the interruption of a neural pathway or its development or the loss of the relevant neural input? Is brain plasticity a generic phenomenon, so that the synaptic reorganization involved in acquisition of knowledge or new memories is the same as that involved in remapping of brain areas? How does the brain reintegrate sensory input into a single image or sound or smell after it is processed by a number of different areas? How does the brain integrate perceptions from separate modalities into one? How does a smell evoke emotion?

It is in this area, more than perhaps any other area in biology, that differences between molecular and organismal biologists' interests most diverge. There has been incredibly rapid progress in identifying genetic and molecular aspects of neural development and arrangement and of the genetic and neural wiring of various senses that must integrate information from the environment. But lists of transmitters, receptors, and gene expression cascades do not bring us much closer to understanding the phenomenon, or perhaps better the experience of perception, than philosophers do.

Was this article helpful?

0 0

Post a comment