Robotic algorithms help pose life's big questions

In a soaring room on the second floor of the Contemporary Jewish Museum in San Francisco, a camera is following your every move.

It's not a security camera, though. It's a customized cam that instantly identifies your position in the room, and based on that, directs audio to you.

The audio, in this case, consists of questions, and lots of them, projected over the extensive sound system: "Can we talk?" "Do you love me?" "Is that all there is?" "When does life begin?" "Do you feel lucky?" "How big is the step between believing and knowing?"

Such queries--gleaned from Jewish scripture and popular culture, among other places--are at the heart of "Are We There Yet? 5,000 Years of Answering Questions with Questions," a reactive sound installation meant to explore the history and future of curiosity in the context of Jewish tradition and beyond. The exhibit opens March 31 and runs through July 31.

"As you move through the space, the camera detects your motion and the sound is delivered exactly to where you are, so you're almost swimming in an auditory sea of questions and voices," Gil Gershoni, one of two local artists who conceived of the exhibit, told CNET. "But it is very specific to your behavior in the space and creates a very personal experience."

As visitors proceed farther into the Yud Gallery, for example, the questions they hear through one of 20-plus speakers on the floor and overhead become more contemplative. The queries are presented in voice-overs by more than 30 people, including NPR personalities.

"The questions are intended rhetorically. It's not about answering them, but about considering the questions and what perceptions and ideas those lead to," said Ken Goldberg, an artist and professor of robotics at UC Berkeley who created the exhibit with Gershoni. "It's important for all of us to keep asking questions. Like friction, they provide the resistance that pushes us forward."

Building on the open-source library for the OpenCV vision-tracking system, Goldberg extended machine learning algorithms for robots to program the small FireFly CMOS camera (about the size of a Canon PowerShot) so it tailors audio in real time depending on a visitor's position in the gallery. Gershoni, creative director of brand consultancy Gershoni Creative, designed the audio experience with engineers from Meyer Sound, the Emeryville, Calif.-based company that developed the sound setup for the Bejing Olympics and Cirque du Soleil.

Questioning is considered central to Judaism, whose primary source of Jewish religious law, the Talmud, poses questions and arguments, not answers. "Are We There Yet?" opens shortly before the Jewish holiday of Passover, when children traditionally ask the Four Questions at the seder table and questions and answers play a prominent role in the retelling of the story of Jews' liberation from slavery in ancient Egypt.

Before then, and through the duration of the exhibit, the public is invited to submit questions online (more than 1,000 have come in so far) to be used in the installation. Visitors to the gallery will also be able to see the range of questions proposed (and pose new ones) via custom iPad-based kiosks with video animation.

"Social technologies are a part of our palette," Gershoni said. "Combining technology with tradition is at the core of this installation."