Vocal Joystick controls PCs for those with hand injuries
University of Washington software project is aimed at, among others, Iraq war veterans who have returned with injuries preventing them from using computers.
SEATTLE--For many Iraq war veterans who have returned home with debilitating injuries that, for example, make it impossible to use their hands, doing anything on a computer is a hopeless task.
But a research project under way in the University of Washington's electrical engineering, linguistics and computer science departments could be the latest tool at such veterans' disposal, as well as for anyone who lacks the full use of their hands.
The project, known as the Vocal Joystick, is designed to allow someone to control a computer cursor using nothing more than their voice.
University of Washington graduate student Jon Malkin, who spoke at thehere Saturday, described it is an extension of speech recognition technology.
It works by having a user train the Vocal Joystick software with his or her voice.
"We can do a lot with that," Malkin said. "Speech is a very complex signal."
Indeed, he explained that the software allows a user "four degrees of freedom," meaning that the cursor can be moved up, down, left and right, all with different vowel sounds.
For example, to move left, the user would sound out the "ee" tone in "feet." To go right, it would be the "aw" sound in "law." Hitting the equivalent of the mouse button would be the "u" sound in "but."
It might be tempting to think that only the most basic tasks are possible with this kind of software, but Malkin said that one artist who has used the Vocal Joystick had employed it to paint a picture of Mount Fuji that he then showed the audience.
The picture was very good, something that many people with full use of their hands would never have been able to create.
He said the artist had created the picture in just three hours.
Malkin also demonstrated the software in real-time, showing how it is used in conjunction with a simple game where a player controls a fish swimming around trying to catch other fish.
He proceeded to sound out vowel after vowel, and sure enough, on-screen, his fish moved around dexterously, chomping up snack after snack. The Gnomedex crowd went wild.