Music for Your SkinI settle into an orange camp chair that has been stripped and fitted with 16 small, round voice coils, or speakers without loudspeaker cones. Carmen Branje hands me a pair of noise canceling headphones connected to his smartphone and launches a white noise app with instructions to jack up the volume as high as possible. Soon I feel as if I’m underneath a rushing waterfall. Branje hits play on a nearby computer and the chair begins to vibrate, though I can no longer hear the whir. The beat plays fast across my skin, the buzzes fluctuating from soft to intense as they flit up and down my back.
After a few minutes, the buzzing stops and I remove the headphones. Happy or sad? Branje asks. Happy, I respond. Definitely happy.
Audio-visual technologies have long dominated the media landscape, but touch-based, or haptic, technologies, could soon become more than a novelty. As far back as the 1950s, renowned psychologist and tactile researcher Frank Geldard noted that our narrow focus on the ears and eyes ignored other perfectly good channels of communication. The skin, he wrote in 1960, “is a good break-in sense: cutaneous sensations, especially if aroused in unusual patterns, are highly attention-demanding.”
Nowadays, with so many gadgets clamoring for the attention of our ears and eyes, using the skin as an alert system has started to gain traction. Cell phones buzz. The handheld controller in Wii vibrates when a user knocks out his opponent in a mock boxing match. General Motors recently installed vibrators inside the seats of its luxury cars to alert drivers when somebody enters their blind spot or when they’re drifting too far to one side. Researchers are looking at placing touch sensors along the body to improve bowing technique on the violin, facilitate rehabilitation after an injury or stroke, allow coaches to direct players on the field without yelling, and even help astronauts stay oriented in space.
But here, at the glass-walled Inclusive Media and Design Centre at Ryerson University in Toronto, Canada, the goals are loftier. Branje, who was a week shy of defending his doctoral thesis when I visited, believes our sense of touch can do a lot more than receive alerts. Cell phones vibrate at a single frequency, and so does the buzzer under one’s butt in a Cadillac. But Branje, himself a musician, thinks touch can be used to mimic an octave or the feel of a melancholic song. Someday, he says, maybe we’ll have a new genre of “music” based not on sound but touch. “Instead of an mp3,” he says, “I would send you a vib [vibrational] file.”
Hearing Speech Through SkinNobel laureate Georg von Békésy, a biophysicist at Harvard University, frequently lamented that those studying hearing, like himself, rarely researched the skin. Békésy’s research, inspired by his desire to understand difficult to access parts of the ear via the skin, showed that acoustic elements like pitch, loudness, and rhythm all had touch-based equivalents.
Anatomy backs up his discoveries. The skin is underlain with four mechanoreceptors that each respond to different forms of touch, such as a light tap, pressure, or pain. But in the 1980s, researchers found that those same receptors could respond to—or “hear”—different frequencies. Compared with the tens of thousands of mechanoreceptors in our ears, known as hair cells, the resolution of the tactile system is terrible. But their existence illustrates striking similarities between the two systems.
Much of our early understanding of the skin’s ability to receive sound comes from a line of research that blossomed in the 1970s and ’80s: enabling deaf people to “hear” the vibrational patterns that make up speech through their skin. That idea first emerged at a deaf school in the 1920s, says Janet Weisenberger, a psychologist at Ohio State University in Columbus, and an early researcher in the field. At that time, researchers essentially affixed miniature loudspeakers to user’s hands and fingers, but because the skin can’t feel anything above 1,000 hertz—which is where we distinguish among different vowels and consonants—users were able to detect that someone was speaking, but not comprehend what was being said.
Weisenberger and her team have studied a 16-vibrator device that could circumvent the skin’s frequency limitations by utilizing its abundant surface area. For instance, Weisenberger arrayed the device across users’ arms. A high-frequency note like the sound of the letter “s” would buzz near the elbow while a lower frequency note like “oo” would buzz near the wrist. The name Sue would thus vibrate in quick succession at the elbow followed by the wrist. In that way, people could be trained to recognize the patterns that went along with different vowels and consonants, Weisenberger says. The device “transformed frequency into location.”
That finding, coupled with others such as helping users distinguish between words that appear identical on the lips (look in a mirror and say pat and bat) significantly improved the accuracy of lipreading, which alone enables a user to understand a frustratingly low 30–60% of a conversation.
But before the device could reach prime time, another technology soon upended that work. Cochlear implants, which rose to prominence in the early 1990s, reroute sound from the damaged hair cells in the inner ear directly to the auditory nerve. Today, deaf individuals with cochlear implants have greater access to speech and sound than ever before. Funding and interest in haptic speech soon dried up. Yet the information gleaned from that work could serve another purpose—giving the deaf greater access to music.
A closer look at the voice coils on the Emoti-ChairBranje's custom keyboard activates the voice coils on the Emoti-Chair.full article:
http://www.pbs.org/wgbh/nova/next/body/haptic-hearing/