Make music from pictures with Reactable

Not familiar with the Reactable music instrument? Prepare to be enlightened.

The Reactable was first invented by the Music Technology Group at the Universitat Pompeu Fabra in Barcelona. It is a translucent table that responds to triggers placed upon it, such as blocks or fingertips. The blocks (called tangibles) are placed and manipulated in such a way that sounds play from a virtual synthesiser.

Australian musician Mark Forester, who performs under the name Surveillance Party, has created a Reactable of his own using a modified Sony PlayStation Eye camera underneath the surface of the table. Using the open-source Reactivision software, which plots a data stream of position, x, y, acceleration and fiducial number from the camera feed, Forester took 12 months to program a patch that Pure Data (another piece of open source software) interprets the camera data and sends it to Ableton Live, triggering the music.

As you can see in the video below, the patterns on the blocks are reasonably complex. "According to the Reactivision creators, these patterns are called 'Fiducials' and were created using a process of simulated evolution," Forester said, "which apparently is shorthand for a left-heavy depth sequence with random mutations."

Should you want to try making a Reactable of your own at home, a program called Fidgen lets you create these patterns. Forester was inspired to create a Reactable after finding other projects weren't meeting his expectations. "The first time I saw this technology I was jolted emotionally, intellectually, spiritually and it took a few days for me to settle," he said. "I saw a control mechanism which would allow me to perform live any music I could make. An instrument which could be used to DJ in a club, or control a vision application, or put in a museum and played by strangers ... it should be playing beautiful, inspiring music. That's my game plan."

Find out more about the Reactable project at the university website or watch Forester's version in action.