The sound (and sight and feel) of music for the deaf

(Original Link - http://www.theglobeandmail.com/news/technology/science/the-sound-and-sight-and-feel-of-music-for-the-deaf/article1792763/)

Frank Russo helps make music for the deaf.

Working with a team of researchers, the Ryerson psychology professor invented a chair that allows deaf people to feel music through vibrations. He also works with both deaf and hearing musicians to compose music that focuses on vibrations and vision rather than sound.

Prof. Russo, a music cognition expert who also sings and plays guitar, will discuss music without sound at the TEDx Talks in Toronto Thursday. The conference’s tagline is “ideas worth spreading.”

Your talk will be on experiencing music without sound. Tell me more.
 
I plan to talk about the other modalities – or the other senses – and whether or not we can experience music through these other senses. This is interesting from a scientific perspective. It also has some interesting practical and artistic implications when we’re considering music experienced by the deaf.
Performers do things when they’re performing that convey emotion and these things can be seen. So, for example, when a performer is performing something that is melancholy, their movements are melancholy. By movements, I mean their facial expressions, the way that their body moves, the way that their hands move. There’s really a lot that can be seen that conveys important structural and emotional information about music. There’s [also] a long history of the deaf experiencing music through vibration.

Legend has it that in his later years, a deafened Beethoven cut the legs off his piano to feel the vibrations through the floorboards. How do deaf people experience music and how does this inform your work?
 
Deaf culture is extremely visual and it also involves the body, more prominently I would say than oral cultures. So their experience of music, maybe not surprisingly, is informed by what they see and what they feel. There’s this long history of feeling music. For example, there’s a famous percussionist, Evelyn Glennie. She’s deaf and she talks about experiencing music through her body. So she’ll perform without shoes so that she can feel the vibration through her body.

You and a team of researchers at Ryerson developed the emoti-chair. What is it and how does it work?
 
The emoti-chair is a sensory substitution technology that’s designed to take sound and present it to the body as vibration. You can put your hand on a speaker and you can feel the vibration because all sound emanates from some form of vibration. The challenge, though, with touching a speaker or even touching a musical instrument is what we call perceptual masking. Perceptual masking occurs in vibration when the lower frequency vibrations dominate the higher frequency vibrations. So all we feel is the thump, thump, thump. So what we’ve done in the emoti-chair is separate out the frequencies and present them to different parts of the body. We’ll take the high frequencies and we’ll present them to the upper part of the back. We’ll take the lower frequencies in the music signal and we’ll present them to the lower part of your back.

You’ve held a couple dozen concerts for deaf and hard-of-hearing people with the emoti-chair. What are the concerts like?
 
It’s really evolved. We’ve gone from taking prefabricated music that’s been constructed for hearing ears and have translated it into deaf music. We are now doing something entirely different, where from the conceptualization of the music we’re thinking about this as a vibe track or a piece of music that’s primarily for vibration and vision, not sound. So that opens up all sorts of interesting artistic possibilities for the deaf and hearing community.

It sounds like you’re almost creating a new art form of music without sound.
 
That’s what we like to think, yeah. And we actually are putting on a series of workshops across the country where we’re exploring this. We did one in Vancouver last June. We’re going to do the next one at the Banff Centre for the Arts next spring. At these workshops, we’re trying to bring together music performers or composers that want to work on this new art form, on developing something that’s music-like but has this reallocation of the sensory priorities so that vibration and vision are in the foreground.

Do people who experience music without sound also experience the emotion that is so much a part of music?
 
Absolutely. We have been doing some research in the lab along those lines. And yes, there’s a great deal of agreement between the emotion experienced by a deaf individual and a hearing individual.

Scientists Show How Tiny Cells Deliver Big Sound In Cochlea

(Original Link - http://www.sciencedaily.com/releases/2009/10/091022114319.htm)

Deep in the ear, 95 percent of the cells that shuttle sound to the brain are big, boisterous neurons that, to date, have explained most of what scientists know about how hearing works. Whether a rare, whisper-small second set of cells also carry signals from the inner ear to the brain and have a real role in processing sound has been a matter of debate.

Now, reporting on rat experiments in the October 22 issue of Nature, a Johns Hopkins team says it has for what is believed to be the first time managed to measure and record the elusive electrical activity of the type II neurons in the snail-shell-like structure called the cochlea. And it turns out the cells do indeed carry signals from the ear to the brain, and the sounds they likely respond to would need to be loud, such as sirens or alarms that might be even be described as painful or traumatic.

The researchers say they've also discovered that these sensory cells get the job done by responding to glutamate released from sensory hair cells of the inner ear. Glutamate is a workhorse neurotransmitter throughout the nervous system and it excites the cochlear neurons to carry acoustic information to the brain.

"No one thought recording them was even possible," says Paul A. Fuchs, Ph.D., the John E. Bordley Professor of Otolaryngology-Head and Neck Surgery and co-director of the Center for Sensory Biology in the Johns Hopkins University School of Medicine, and a co-author of the report. "We knew the type II neurons were there and now at last we know something about what they do and how they do it."

Working with week-old rats, neuroscience graduate student Catherine Weisz removed live, soft tissue from the fragile cochlea and, guided by a powerful microscope, touched electrodes to the tiny type II nerve endings beneath the sensory hair cells. Different types of stimuli were used to activate sensory hair cells, allowing Weisz to record and analyze the resulting signals in type II fibers.
Results showed that, unlike type I neurons which are electrically activated by the quietest sounds we hear, and which saturate as sounds get louder, each type II neuron would need to be hit hard by a very loud sound to produce excitation, Fuchs says.

The cell bodies of both type I and type II neurons sprout long filaments, or axons that head to the brain, and some others that connect to sensory hair cells. Unlike the big type I neurons, each of which make one little sprout that touches one sensory hair cell in one spot, the type II cells have projections that contact dozens of hair cells over a relatively great distance.

"Somewhat counter-intuitively, the type II cell that contacts many hair cells receives surprisingly little synaptic input," Fuchs says. "In fact, all of its many contacts put together yield less input than that provided by the one single hair cell touching a type I neuron."

Fuchs and his team postulate that the two systems may serve different functional roles. "There's a distinct difference between analyzing sound to extract meaning -- Is that a cat meowing, a baby crying or a man singing? -- versus the startle reflex triggered by a thunderclap or other sudden loud sound." Type II afferents may play a role in such reflexive withdrawals from potential trauma."
This study was supported by the National Institute on Deafness and Other Communication Disorders, and a grant from the Blaustein Pain Foundation of Johns Hopkins.

Authors on the paper are Fuchs, Weisz and Elisabeth Glowatzki, all of the Center for Hearing and Balance and the Center for Sensory Biology, Johns Hopkins University School of Medicine.