New Scientist, 11 August 2001
knows there are five basic senses. But try separating them one from the
other in your daily life and suddenly they don't feel so distinct.
a banana, for instance, and try to taste it without smelling it and
experiencing that banana-y texture on your tongue. Can you really just
taste, or must you sometimes taste-smell-feel? Try talking to your
lover. Listen to what is said without watching the mouth move or
feeling the caress of a hand. Can you simply hear, or is there always
an element of hear-see-touch? Even on the phone, can you hear a voice
without imagining a face? Hard, isn't it?
The prevailing view of
the brain still holds that there are five separate senses that feed
into five distinct brain regions preordained to handle one and only one
sense. The yellowness of the banana skin, the texture of its flesh, its
smell and taste-each of these elements is parcelled up and analysed in
isolation. Some theories of consciousness suggest that these dedicated
brain areas somehow stamp each sense with a unique "feeling". Then, the
theory goes, the brain pastes the fragments back together, calls on
memory to give it a name and recall what it's for and, voila, a banana.
perhaps it's time for a radical rethink of how the brain works. Tasks
we've long assumed were handled by only one sense turn out to be the
domain of two or three. And when we are deprived of a sense, the brain
responds-in a matter of days or even hours-by reallocating unused
capacity and turning the remaining senses to more imaginative use. All
this begs the questions: are the senses really so segregated? Are they
separate at all? Indeed, is it possible that our senses are
continuously developing and merging so that each one of us has our own
private view of the world?
It might be a big shift in thinking,
but it began with a simple finding-the discovery of "multisensory"
neurons. These are brain cells that react to many senses all at once
instead of just to one. No one knows how many of these neurons there
are-maybe they are just a rare, elite corps. But perhaps there are no
true vision, hearing or touch areas dedicated to a single sense, after
all. Perhaps all the neurons in our brains are multisensory-and we
mistakenly label them "visual" or "auditory" simply because they prefer
one sense over the others.
That's the view of Alvaro
Pascual-Leone at Harvard University. He made a splash five years ago
when he showed that people who were born blind use the visual cortex
when they read Braille. He wondered if rather than lie idle, parts of
the brain meant for seeing just started helping out with touching. His
more recent work has convinced him that not only blind people but
everyone has the capacity to swap senses if they have to. He thinks
that the brain is much more versatile than most researchers would have
To test the idea, Pascual-Leone blindfolded healthy,
sighted volunteers for five days running, taught them Braille and
watched how their brains responded. He even fitted their blindfolds
with photographic paper-just to be sure volunteers weren't tampering
with them. Before, during and after the blindfolding, they had a series
of brain scans while they were set different tactile and auditory
tasks-feeling either Braille characters or brush strokes on their
fingertips and listening to tones or word fragments. Before the
blindfolding began, the "visual" areas were not switched on by the
touching and hearing tasks. But as the week wore on the visual regions
became more and more involved in routine touching and hearing.
a person isn't seeing, Pascual-Leone found, parts of the "visual"
cortex are roped in to help out in tasks involving other senses. In
fact, the newly recruited regions soon become indispensable. When he
tried temporarily disrupting the workings of the visual areas, using a
technique called trans-cranial magnetic stimulation, or TMS, the
blindfolded volunteers found it hard to read their Braille.
the blindfolds off for just a day, though, was enough to undo the
changes; suddenly touching and hearing tasks no longer triggered visual
areas, even though volunteers were blindfolded again briefly for the
scan. "Removing the blindfold and being exposed to the seeing world for
12 to 24 hours is sufficient to revert all changes induced by the five
days of blindfolding," says Pascual-Leone.
What was astonishing
was how quickly the brain seemed able to recruit new areas and equally
effortlessly reverse that process. It was far too quick to be the
result of new connections forming from scratch reasoned Pascual-Leone.
"It must be assumed," he says, "that tactile and auditory input into
the 'visual cortex' is present in all of us and can be unmasked if
Pascual-Leone now feels the brain is
not organised into "visual" and "auditory" and "tactile" regions at
all. Instead he thinks it is split into units that have specific jobs
to do or particular problems to solve-calculating distance, for
example, or timing intervals. These problem-solving units simply use
the best information available. Sometimes they may prefer certain
senses to others, based on how suitable they are for the assigned
computation, and sometimes they may use more than one, if that helps.
Vision, for instance, might be the preferred way to judge distances.
But if you can't see, hearing or touch can certainly fill in.
preference of a particular problem-solving unit for a specific sense
may explain the notion of sense-specific regions, he says. Just because
an area tends to call on vision doesn't mean it can't process other
senses, only that it may not bother if its first choice sense is on
hand. This may have tricked neuroscientists into thinking that the
brain is structured in parallel, segregated systems processing
different types of sensory signals, says Pascual-Leone.
some good evidence that the brain can mix up the senses to solve
particular problems. One of the main benefits of sensory integration
may be better clarity and detection, says Barry Stein, at Wake Forest
University in Winston-Salem, North Carolina, one of the first
researchers to identify the brain's multisensory capabilities. Even
weak signals should be taken seriously if they're picked up by more
than one sense.
We are, for example, much more sensitive to a
chemical when we combine smell and taste. Pamela Dalton, at the Monell
Chemical Senses Center in Philadelphia, asked 10 people to smell
benzaldehyde, a cherry-almond odour that has no taste, and to taste
saccharin, a sweetener that has no smell. Before each testing session,
she worked out the point where each volunteer could no longer detect
each substance and prepared even weaker samples. Then she asked them to
slosh the solution around in their mouths and sniff the odour at the
same time. Combining taste and smell made both substances much more
apparent, she found. "Ten minutes before, they hadn't been able to
detect it," says Dalton.
A brain combining senses can also make
better sense of ambiguous information. David Lewkowicz at the New York
State Institute for Basic Research in Developmental Disabilities on
Staten Island shows this nicely with a visual image of two balls moving
from opposite sides of a screen, merging briefly in the centre, then
continuing along their merry ways (see "Brain Games"). But when a beep
sounds at the moment the two balls merge, what you see changes
completely. Now, instead of passing through each other and continuing
along the same trajectory, the two balls bounce off each other and
return to the side they came from.
Combining hearing with vision
can lead us to draw different conclusions about what we've seen too. A
single flash of light, can appear to be two flashes when it coincides
with two beeps, says Ladan Shams and her colleagues at Caltech in
Pasadena. Even when we know there is just one flash, we can't help
perceiving it as two. Apparently the brain won't let us draw
contradictory conclusions from two different senses.
scientists are discovering that even everyday activities may actually
make use of more than one sense. Consider the task of running your
fingers over a pattern of raised ridges and deciding in what direction
they are running. What sense do you call upon? Most of us would guess
the obvious: touch. But a group at Emory University in Atlanta has
demonstrated that in perfectly normal people parts of the "visual"
brain are also essential for perceiving touch.
They started by
scanning people's brains to see what regions were activated when they
were trying to decide the orientation of some grating patterns on a
touch pad. They found that a part of the brain that's involved in
recognising objects by sight was active while people felt the gratings,
even though they couldn't see them. "What excited us was what our
subjects told us," says Krish Sathian, a lead member of the team. "When
they were doing the tactile task, they were actually visualising in
their mind's eye the orientation of the grating."
imagery just provide a convenient aid, or was it essential to the task?
To find out, they used the TMS technique to disrupt the activity in the
"visual" region the volunteers had been using. Suddenly, their
volunteers could no longer tell the direction of the pattern.
researchers concluded last year in the journal Nature (vol 401, p 587)
that the "visual" cortex is closely involved in certain tactile tasks.
They claimed it was the first time that visual processing was shown to
be instrumental in ordinary tactile perception. But Sathian admits that
the activated region may not really be visual at all. It could be a
part of the brain that helps us visualise what's being touched. "We
certainly can't rule out that what we're seeing is multimodal
processing in an area previously thought to be just visual," he says.
bold interpretation, that the brain is organised by task rather than by
individual sense, is by no means the accepted one. Even most scientists
who study multisensory processing consider it extreme. "At least some
areas are exclusively unisensory," says Sathian. There's very clearly a
primary visual cortex with strong inputs from the eye, he says, and a
primary somatosensory cortex getting information from the body. But
that's not to say that the map of the brain is static-far from it. New
multisensory areas are being found all the time. "The boundaries are
being pushed back," says Sathian, "just not pushed back all the way."
boundaries were seriously tested by an experiment that involved
"rewiring" the brains of ferrets. The findings called into question the
well-guarded notion that certain brain areas can only dedicate
themselves to certain tasks. They suggest that, although the brain may
tend to develop in a particular way, with vision processed at the back
of the head and hearing on the sides, it doesn't have to be that way.
group at MIT in Boston wanted to know how much they could override
innate developmental pathways. "If we put the retina into the auditory
cortex, will it see?" asks Sarah Pallas, a member of the team, now at
Georgia State University in Atlanta. The researchers surgically
rearranged one brain hemisphere in a handful of newborn ferrets, so
that the nerves from the retina, which normally go to the visual
thalamus and then on to the visual cortex, now connected to the
auditory thalamus and eventually to the auditory cortex.
their surprise, they found that the auditory cortex on the rewired side
arranged itself like a visual cortex: the cells showed selectivity for
orientation and motion, and they encoded a two-dimensional map of
visual space. The rewired animals also seemed to behave perfectly
normally. Using only the untouched hemisphere the researchers trained
the animals to go to a food spout on one side of a test room if they
heard a sound and one on the other if they saw a light. Amazingly, even
after the visual cortex on the healthy side was completely destroyed,
the animals found their way to the food.
"We were able to turn
the auditory cortex into a visual cortex," says Pallas. "Maybe they
couldn't recognise their grandmother with that, but they certainly
could detect light." In fact, the young ferrets seemed so normal that
the researchers had to mark them to tell them apart from their siblings.
experiment revealed just how multimodal the brain may be. The amazing
rewired auditory cortex was not only seeing-it was hearing at the same
time, Pallas told a meeting of multisensory scientists in New York last
autumn. Though the finding has not yet been published, she said that
preliminary testing showed that the rewired auditory cortex was
responding well to sound.
What's more, the study shows that what
goes into the brain can have a lot of influence on how it's ultimately
organised. Although some parts of the brain may be predisposed to
become one thing or another, the rewiring shows they aren't
predetermined. "Sensory inputs can influence the regional identity of
the cortex," says Pallas.
But how far does this go? We can
fairly assume that people deprived of sight early on will have their
brains wired up differently from people who see. But what about someone
who has been nearsighted since birth-could that person have a quite a
different brain from someone who's experienced the world through
sharper eyes? Is someone born into the high rises of Hong Kong wired up
differently from a person growing up in the Gobi desert?
thinks that, both at the functional and the anatomical level, our
brains are quite unique. "Blind people are not experiencing the world
like a sighted person with eyes closed," he says, "but rather, they
have a dramatically different world representation and hence
consciousness." Indeed, maybe each of us has our own very personal take
on the world, sensed by our own unique brain.
Alas, we only know
how it feels to be ourselves, so it's impossible to know. And we can't
ask those ferrets whether they were really seeing, or somehow hearing
the light. It makes you wonder all over again about bananas-is the
divine yellow fruit the very same to you as it is to me? Probably not.