Graphics conference plays up interaction
Immersive technologies that let people express ideas or input data through ordinary physical actions are one of the dominant themes at the Siggraph computer graphics conference.
Body-Brush, an application being developed by Young and his colleagues, lets people compose 3D paintings or perform electronic music through movement. Raise a hand and musical pitch increases; spread one's arms and volume surges. In drawing mode, streaks and lines appear on a wall-mounted screen to represent a person's running pattern. The speed, gait and acceleration of the walker affect the tone and look of the images.
"We've talked to psychologists who want to use it for art therapy," Young said. The object tracking part of the technology could also be used for security purposes.
Immersive technologies that allow individuals to express ideas or input data through ordinary actions is one of the dominant themes at Siggraph, a computer graphics conference taking place in San Diego this week.
"Interactive technologies are probably where computer graphics were 20 years ago," said Josh Strickon, an Apple Computer senior engineer and chairman of emerging technologies for the conference.
The Massachusetts Institute of Technology and Sony, meanwhile, are exhibiting applications that compose music with limited data input--handwritten lines on a computer screen or a few notes from a synthesizer.
Some compositions created through MIT's HyperScore, such as "Creepy Raindrops" created by 10-year-old Chelsea O'Hara, were performed by the BBC Scottish Symphony Orchestra, among others.
"They were able to write coherent pieces of music that orchestras were able to perform," said Mary Farbood, who developed the software at the MIT Media Lab. "Creepy Raindrops" was "interesting because it is an atonal piece written by a 10-year-old," she added, although the favorite of the MIT staff was "Attack of the Headless Chickens" by 14-year-old Fiachra MacOireachtaigh.
The push toward interactivity is part of an overall effort by companies and universities alike to find more intuitive ways to control technology. Keyboards, after all, can't be used everywhere, and most games and applications still require people to master a host of complex instructions. Eventually, technologies such as voice and handwriting recognition, or motion tracking, will need to be perfected, advocates say.
In addition, existing microprocessors now contain enough computing power to drive these kinds of applications efficiently.
Sensor nation
Currently, researchers are mostly trying to figure out which techniques work best for which particular applications. Body-Brush, for example, depends on infrared light, Young said. Infrared light is directed to a specified area in a room, and cameras and sensors then monitor where and how the light is reflected.
The reflection data subsequently lets the computer track objects moving in the lighted region and interpolate those into graphical images or sounds.
Canesta, a San Jose, Calif.-based start-up, is using infrared light in a similar fashion in a virtual keyboard for cell phones expected to come out in 2004.
Meanwhile, Thermo-Key, which is under development at the University of Tokyo, lets computers home in on heat. With Thermo-Key, a speaker can get a computerized camera to insert live video of himself or herself--but not a moving balloon or other inanimate object in the same region--into an ongoing presentation.
ElectAura-Net from Japan's NTT and NTT DoCoMo, meanwhile, incorporates a person's body into the tracking process. Electrical current travels from a specialized floor to a handheld in an electrical field that wraps around the person holding it.
"Your body becomes a conductor," said Manabu Sakurai, marketing manager for NTT advanced technology.
Such tracking technologies have their potential pluses and minuses and, of course, create dire privacy implications. In Access, a multimedia art project created by Marie Sester, a computer-driven spotlight automatically tracks random passers-by.
In a trial in Tokyo with unsuspecting pedestrians, most people tried to ignore that the spotlight was following them around. "Then they would turn around when they were out of range to see if someone else would get hit," she said. One elderly woman ran away shouting.
Things went better at a demonstration during a party thrown at IBM headquarters where attendees were briefed on what the roving spotlight was doing and had a few glasses of wine, she noted.
The musical applications, meanwhile, are not sensor driven. Instead, they depend on statistical models, a favorite of the artificial intelligence community. Sony's Continuator will absorb pieces of Bach or random keyboard strokes and then correlate the data against its own databases to finish and/or create musical phrases in the same style, said Francois Pachet, the head music researcher at the Sony Computer Science Lab in Paris.
Similarly, HyperScore "understands" keyboard keystrokes and handwritten lines because of familiarity with similar input.
"Perhaps the hardest part was finding the right mapping of graphics to music," MIT's Farbood said.
New Age monitors
The week-long conference is also playing host to companies demonstrating novel monitors.
Finnish start-up Fogscreen has come up with a system that lets images be displayed on a curtain of vapor. Vapor is shot out of jets in a ceiling unit and then prevented from escaping or curving via forced air currents coming out of the same ceiling unit. The image eventually dissipates as the air currents weaken, but the picture retains its shape for about three feet.
"If you can't keep it flat, you can't project an image on it," said Timo Rakkolainen, a company engineer and brother of the company's founder. Fogscreen hopes to sell the system to malls as an advertising kiosk.
At the other end of the spectrum, ARC Science Simulations is marketing OmniGlobe, a spherical display and system. The displays, which are globes that contain an internal hemispherical mirror for blowing up images from a computer, measure 60 inches to 80 inches in diameter and can display maps of planets, global magnetic fields or other naturally curving items on a sphere. The idea is to improve on flat maps, ARC founder Thomas Ligon said.
One of the more popular applications for the displays, the Paleo-Earth project, shows how the continents have moved and changed in shape over time due to continental drift. The Paleo-Earth animation contains 3,600 images, which morph into each other as the mouse is moved and represent 600 million years of tectonic movement.
The globes, which can cost $98,500 and more when fully configured, are largely sold to museums and government agencies, Ligon said.