Imagine walking on a Calcutta street on a rainy evening, reading a movie billboard, talking to your friend on your cell phone and sipping a drink all at the same time. Suddenly you encounter an open manhole on the sidewalk and you stop instinctively to avoid a mishap. Then you see that the footpath ahead is flooded and you step off the curb, watch for approaching traffic and then cross the street. All the while, however, you continue talking on the phone, enjoying your drink and then turn around to stare at a beautiful girl crossing the road.

This is a common day occurrence in any Indian city. But from a processing perspective it’s pretty complex for the brain. Not only are we using all our regular senses but are also putting our proprioceptive sense (the ability to sense the position, location, and movement of the body) and the somatosensory sensations (senses like warmth, cold, vibrations, pressure, pain, etc) to full use.

The human brain can process five senses — sight, hearing, touch, smell and taste. Touch is actually one component of the proprioceptive sense, which also includes the sense of balance, and all the other somatosensory sensations. While a common man may forget to mention this sense, the proprioceptive sense is very critical to common activities and, unlike our eyes or ears which we can close or cover, it can not be “turned off”.

Even when we are engaged in another task, like reading a newspaper, we can still pick up our coffee mugs with little effort. However, this natural action will be a lot more difficult if we have to get our coffee mugs by typing commands on a keyboard or using a mouse to point at it.

Modern technology has embarked on a quest to make this a reality. Human-centred computing and human-centred interfaces now seek to improve the interaction between humans and man-made systems.

A team of researchers at the Florida Institute for Human and Machine Cognition hopes to change this for good. Led by Dr Anil Raj, the team at the institute and other scientists are trying to reduce the workload on our traditional senses by waking up additional senses such as tactile or auditory senses, which often lie dormant in our day-to-day interactions with man-made systems. “Our research seeks to develop methods to interact with cars, aircraft, databases, process control, etc, in ways that will seem as intuitive as drinking your coffee while reading a newspaper,” says Dr Raj.

Dr Raj, Dr Paul Bach-y-Rita of the University of Wisconsin and scientists at Wicab Inc, a US company that makes the BrainPort™ tongue display system to aid blind people and those with balance disorders, are exploring ways of providing high resolution cues through the tongue.

Dr Bach-y-Rita has chosen the tongue for this noble mission as it does not have a layer of dead skin and, therefore, allows much lower current and voltage to be used for any stimulation. In addition, the tongue has one of the most densely packed nerve endings in our bodies. Therefore the researchers can use the BrainPort™ to provide higher resolution electro-tactile signals on the tongue than on other parts of the body. Lastly, the tongue is very close to the brainstem and this means that signals can be perceived with minimum temporal delay.

The system aims to extend normal senses without interfering with existing sensory information. With this system, fire-fighters can “see” through smoke while search and rescue divers could navigate through murky waters by directly using the tongue for infrared imaging or navigation. “It’s just a new way of communicating with someone. With appropriate training any person can learn to transmit and receive electronic signals,” says Dr Raj. “The sensation is similar to the sensation of carbonated beverages on the tongue, but with proper feedback, one rapidly starts to perceive the information rather than the stimulus. Just like you are unaware of light hitting your retina or the movement of the hair cells in you inner ear (you perceive the visual images and hear the sound, respectively, produced by the stimulus), you can learn to perceive through the tongue,” he adds.

But having this additional channel of information could complicate things for an “over-tasked” individual. Therefore much effort has been put to ensure that this new information pathway does not interrupt a user’s attention (unless if necessary in a specific application) and minimises the possibility that the user will miss important information. In other words, this new technology allows us to use multi-sensory displays to improve the ability of people to perceive, understand and act on information in complex and dynamic environments.

Not everybody is impressed though. Karen Hurley, a PhD student of interdisciplinary/ environment Studies at University of Victoria, Canada, says, “Won’t it be great if all this human ingenuity, time and money goes into development, health care, protecting our environment, etc?”

But detractors apart, someday this technology may allow you to drive a car and watch a movie on your personal DVD player at the same time. Till then, please keep your eyes on the road.