Why Sony Imageworks got an Oscar nod for 'Surf's Up'

CULVER CITY, Calif.--One of the things that struck me towards the end of the animated surfing penguin mockumentary, Surf's Up, is that I had forgotten that every bit of water in the film--mainly loads of lovingly rendered surfing waves--was digitally animated.

Even after that realization, I looked at the water in the film and thought the animators had done a remarkable job at recreating one of the things that, like making human hair look realistic, has always been hardest to recreate.

Others must agree, because last month, Surf's Up was chosen as one three nominees for the best animated feature Oscar. And while the film would have to be considered a big underdog, since it, and its fellow lesser-known nominee, Persepolis, are going up against Pixar's juggernaut, Ratatouille, a huge critical and commercial success.

Still, when Sony Imageworks, the Sony Pictures in-house visual effects studio behind the imagery in films like Spider-Man 3, Beowulf and I am Legend, invited me down to L.A. to talk about Surf's Up and the animation wizardry behind it, I readily agreed.

Walking into Imageworks' offices in Culver City, just a couple blocks down the street from Sony Pictures Studios' gargantuan facilities, one of the first things that struck me was how dark and quiet it was. All the better to keep glare off of animators' computer screens and to get work done, I was told.

Soon, I was ushered into a small screening room where the brains behind Surf's Up, producer Chris Jenkins, co-directors Ash Brannon and Chris Buck, and visual effects supervisor Rob Bredow had gathered to talk to me about the film and about those fantastic waves.

Lest you think that creating waves for an animated surfing film starring penguins is a simple job, let me assure you that it isn't.

Imageworks' special sauce begins with its wave animation control system, a proprietary technology the company developed to create the waves for Surf's Up, and which was a modified version of the system it used to make the water effects for the Tom Hanks film Cast Away.

Waves are "essentially modeled one section at a time," said Bredow. "They start flat, turn into a swell, and then flip over and turn into classic crashing waves....We modeled (them) on what real waves would look like."

The film's story, Bredow added, called for three main types of waves: simple spilling breakers, classic tube waves modeled on Hawaii's famous Pipeline, and the kinds of huge waves found at Northern California's celebrated Maverick's.

"We had a bunch of surfers, guys who loved surfing, on the movie," Bredow said. "They'd go through hours of (surfing films) and find waves" they liked and which the Surf's Up team could model the film's waves on.

Looking at what seem like schematic still images of the structure of the waves, one sees what looks like the shape of a big wave with a grid of criss-crossed wire frame lines superimposed on it. The vertical lines, Bredow explained, segment the different sections of the wave, each of which can be controlled individually.

The point with that system is to be able to simulate the rolling effect of a wave crashing, left to right.

And in order to make that happen, the animation system revolves around a series of blue vertical control rings superimposed on a wire frame wave that the animators can "pull" forward.

We "put a ring around key points in each wave," Bredow said. "As we grab the ring and pull on it, or rotate the ring, the corresponding section of the wave will evolve forward and crash."

Bredow said that the basic system for creating waves involves modeling each one from flat water to swell to rolling over to crashing down, and then blending through all those shapes into a single, animated effect.

Then, once the wave animation is created, it's time to add the water texture to it.

For that, Bredow said, his team takes a smooth plane that doesn't look at all like water, and adds many different levels of "noise."

"We simulate thousands of water ripples interacting with each other to simulate the texture," he said. "Basically, we use millions of interfering water ripples to create the wave texture."

Then it's time to add the proper lighting effects, a combination of many different techniques, Bredow explained.

"We made sure the water had all the (right) properties using different photo techniques," he added: "reflection, refraction, and the specular highlights that bloom the right color, and the surface foam on the surface of the water."

Finally, the animators worked on the way light goes through a wave.

Bredow said that when lighting a breaking wave, his team would break a wave down into individual wave "zones," perhaps eleven per wave, and then light each zone individually. They'd use different hues of greens or blues, depending on the need, and voila, a wave.

And as I said above, the results were spectacular. Without commenting on the overall quality of the movie, I will say categorically, that it is beautiful, and the work Bredow and his team did to create the many waves was nothing short of amazing.

One of the choices the filmmakers made when proceeding with Surf's Up, according to co-director Brannon, was to deliberately try to give each wave "character."

"The look of the movie was determined by the water," Brannon said. "We wanted believable water, not necessarily photorealistic, but not stylized, either."

One of the reasons for that is that Surf's Up, as mentioned above, is fashioned as a surfing documentary, focused on Cody, the main character, a young provincial penguin longing to join the glamorous world of pro surfing.

The point, then, was to make the film feel very much like a documentary. And that meant a slightly rougher edge to the texture, including a slightly shaky camera, as well as water on the lens and other such artifacts that wouldn't show up in a normal movie, but which are unavoidable in documentaries.

That's why, when I was watching Surf's Up, I noticed during one surfing scene that there were a couple of drops of water on the lens. At first, it had escaped my attention because it is such a realistic detail that your eye doesn't quite pick up on it. But then I realized that that was intentionally placed there. I had to go back and look again in appreciation of the thoughtfulness behind it.

Another important element in the making of Surf's Up, then, was the incorporation of the live-action camera, something that might not be entirely intuitive in a fully animated movie.

But use a live-action camera they did.

After leaving the screening room and saying goodbye to Bredow, Brannon, Buck and Jenkins, I was taken to Imageworks' layout room, a single room a couple of stories below where, it turns out, the filmmakers shot much of the film.

There, James Williams, Imageworks' head of layout, explained and demonstrated how live-action had been incorporated.

As I mentioned above, the purpose of doing so was to make the texture feel like a documentary. And that meant simulating the kind of slight movements that come when a cameraperson is working with a hand-held camera on location.

To do this, Williams explained, the animators designed a system where they programmed the animation of the many penguins in the film and then turned to their live-action camera, an ancient Sony camera--bought off eBay, no less--that was somewhat like what surfing documentarians would have used a decade or so ago.

The camera was fitted with a special sensor that emitted signals picked up by a grid of hundreds of sensors on the room's ceiling in order to translate the camera's exact real physical movements onto the animated scene.

The result, Williams demonstrated, is that when he moved the camera a little bit from side to side, the animated penguins on screen would shift in the camera's view.

It's an odd concept, and one that took me a little while to understand, but it actually makes a lot of sense, and is a pretty elegant solution to the problem of how to build in the little imperfections in a documentary that the filmmakers wanted to see in their fully digital movie.

All of this was done, said producer Jenkins, so that the desired effect of a surfing documentary felt real to the audience.

"If something's not quite right, even within a tenth of a percent," Jenkins said, the audience sees it. "It has to be just right."

Did it work? Well, as I alluded to above, I think the filmmakers succeeded in making many of the details of their movie work exactly as planned. Things that seemed totally authentic struck me later on, particularly because I realized they were, in fact, totally digital and totally fabricated. And that's the sign of amazing attention to detail.

The film wasn't much of a commercial success, however, perhaps because the story was a little bit predictable and standard. So much of the Imageworks team's labor was missed by the moviegoing public.

But they surely tried hard to make a film that stayed true to the look and feel of the classic surfing documentary. And for that, they deserve their Oscar nomination.

"If you're going for perfection, you fail," said Brannon. "We were going for humanity, and that comes through in the final product and gives it an organic feel."