skip navigation nih record
Vol. LXI, No. 6
March 20, 2009
cover

previous story

next story


NEI 40th Anniversary
A Tongue for an Eye: Device Challenges Conventional Vision

Erik Weihenmayer climbs El Morado south face in Chile.

Erik Weihenmayer climbs El Morado south face in Chile.

Erik Weihenmayer has climbed mountains around the world—the highest peaks, in fact, on every continent. One afternoon a few years ago, however, a climbing wall in a local gym served as the setting of a most memorable journey.

On a typical climb, Weihenmayer is confined to arms-length maneuvers, feeling for holds with his hands. That day in the gym, he scanned the wall with a video camera and his tongue, via a device known as BrainPort. For the first time, he lunged for a hold that was 4 feet away.

Weihenmayer is blind.

“I started learning how to climb like a sighted person,” he says.

A genetic eye condition known as retinoschisis caused him to be visually impaired at birth and completely blind by age 13. In retinoschisis, tiny cysts form within the eye’s delicate retinal tissue, eventually causing its layers to split apart. Neither medication nor surgery can restore sight.

Not a problem for Weihenmayer. In 2001, he became the only blind person ever to reach the summit of Mount Everest. He also skydives, runs marathons and skis, but until his experience with BrainPort, he did all of this while visually guided by others.

On Friday, Apr. 3, Weihenmayer will describe his experiences with BrainPort at the NEI 40th anniversary kickoff. The event will feature a screening of Blindsight, an award-winning documentary chronicling Weihenmayer’s mountain-climbing journey up the north face of Mount Everest with six blind Tibetan teenagers. Researchers will also debut the third-generation BrainPort device, produced with NEI support.

Swapping Senses

Weihenmayer tests the BrainPort device during a game of tic-tac-toe with his daughter, Emma, at home in Golden, Colo

Weihenmayer tests the BrainPort device during a game of tic-tac-toe with his daughter, Emma, at home in Golden, Colo

The scientific concept behind BrainPort originated in the late 1960s. Dr. Paul Bach-y-Rita, a physician and engineer, introduced the idea of sensory substitution—stimulating one sense, such as touch, to take the place of another, such as sight.

“A blind person walking down the street with a cane is using sensory substitution,” says Dr. Michael Oberdorfer of the NEI extramural program. “This person gets spatial and auditory feedback when the cane strikes an object.”

Bach-y-Rita stressed the idea that vision does not come from the eyes alone. The eye receives visual information, converts it to electrical impulses and sends them to the brain for interpretation. With BrainPort, such electrical impulses are sent to the brain by way of nerves in the tongue instead of the optic nerve in the eye.

In the device, a head-mounted camera serves as “eyes” to gather white, gray and black pixels of visual information. A computer translates this information to gentle electrical impulses, which it then sends to an array of electrodes that sits on the tongue of the user. Strong vibrations on the tongue represent white pixels, medium-strength vibrations represent gray pixels and no vibrations represent black pixels.

Learning to use BrainPort is similar to learning a new language, explains Dr. Aimee Arnoldussen, a neuroscientist and BrainPort researcher who worked with Bach-y-Rita until he passed away in 2006. Initially, users must consciously translate the pattern of impulses on the tongue to the idea of an object in space. But as a person becomes fluent in this process, the translation becomes automatic.

“You learn to ignore what’s happening on the tongue because that’s not the most important piece of information,” she says, adding that the most critical step in the process is for the brain to perceive these impulses as a representation of the surrounding environment.

Weihenmayer tests the BrainPort device at his local gym in Golden. Weihenmayer climbs a gym rock wall, with the help of BrainPort.

Weihenmayer tests the BrainPort device at his local gym in Golden.

Weihenmayer climbs a gym rock wall, with the help of BrainPort.


Starting to “See”

While BrainPort does not replace the sense of sight, it adds to other sensory experiences to give users information about the size, shape and location of objects. Arnoldussen says that within an hour, most users can point to different shapes. After a few more hours, they can identify familiar objects and avoid obstacles.

NEI Anniversary Kickoff Features Film

NEI will kick off its 40th anniversary celebration on Friday, Apr. 3 from 1 to 4 p.m. in Lipsett Amphitheater, Bldg. 10 with a screening of the documentary film Blindsight. The event also includes remarks and book-signing by Erik Weihenmayer, a blind mountain climber and author of Touch the Top of the World, and a demonstration of the BrainPort device. The first 250 attendees will receive a copy of the book.

Weihenmayer remembers testing the device for the first time 5 years ago. In just a few minutes, he was able to reach out and grab a rolling tennis ball.

“I thought it was cool how quickly my brain caught on to what I was feeling with my tongue,” he says. “I felt the ball start rolling from the back of my tongue. It started smaller and got bigger.”

The current iteration of the device has a postage stamp-sized tongue array and a camera with a zoom lens. Users can operate it independently with a hand-held controller.

Though testing has remained primarily in the lab, Arnoldussen hopes the device can soon be used by blind people in their daily lives to perform basic tasks such as reading signs and identifying landmarks.

Weihenmayer takes the idea a step further, hoping that all blind children will eventually have access to technologies like BrainPort to help them integrate into society. This, he says, will give them “a better chance of being in the thick of things instead of listening to life go by.” NIHRecord Icon

back to top of page