You rarely see Andy Serkis' face, but you definitely recognize his characters.
His facial expressions and body movements have brought to life some of cinema's most evocative computer-generated characters, including Gollum in the Lord of the Rings trilogy, King Kong in Peter Jackson's 2005 remake, Supreme Leader Snoke in the latest Star Wars films and Caesar in the recent Planet of the Apes movies.
Serkis' skill for chameleon-like transformations has made him the unofficial godfather of performance capture, which involves recording an actor's performance in three dimensions and mapping a digital character over the top. His name is so synonymous with the acting technique that he's co-founded a performance capture studio called The Imaginarium in London. He's also directed a new take on Rudyard Kipling's classic children's tale The Jungle Book in which a galaxy of stars used performance capture to become the story's much-loved characters.
Mowgli: Legend of the Jungle is streaming on Netflix now. Serkis actually began work on his version of the story in 2014, before Disney got in first with its own motion-captured Jungle Book in 2016.
Serkis plays Baloo in his long-gestating, darker take on the story, performing alongside Benedict Cumberbatch as Bengal tiger Shere Khan, Cate Blanchett as the snake Kaa and Christian Bale as the panther Bagheera. I talked to Serkis about the film earlier this year.
CNET: Did some of the stars take to performance capture better than others?
They hadn't worn head-mounted cameras, apart from Benedict. But you just forget you're wearing them, really, after two minutes
Some were more physical than others. Benedict was incredibly physical. Some were stiller -- Peter Mullan, for instance, was like granite, with strict authoritative ruling and stoicism in the way he approached Akela.
How do the computer-generated characters evoke the faces of the actors?
We spent a long time evolving how to fit the actors' physiognomy to the animal. For instance, with Cate Blanchett playing Kaa, we took an image of her face on one end of the spectrum and then the actual design of a python on the other, and we morphed the faces along the timeline to find a sweet spot in the middle, where you could see the actual actor's facial expression and the design of the creature.
None of the animals are photo-real, but they feel real because of the emotions generated in the facial expressions -- and all that comes through in the design, which is bespoke to each actor.
Was matching Cate Blanchett's face with a snake the most extreme transformation?
Yeah, it took the most to humanize that creature because they are sort of featureless. Snakes' jaws just sort of open and shut, they don't have a lot of musculature to makes shapes reflecting human vowels or consonants. Also, the eyes are pretty much on the sides of the head and they're very glassy. We went out of our way to design the brow and the forehead of the snake so that it would hold a more-human pair of eyes. They were actually Cate's eyes, replicated and put into the design, because snakes' eyes are like sharks' eyes: they don't connect with you emotionally.
And Cate's got such extraordinary eyes, I wanted them to be in the design.
Besides the faces, how did you deal with animal shapes and sizes that are so different from the human body?
When we started people said, "How on Earth do you get Cate Blanchett to move like a snake?" We actually had her in a motion-capture suit with the snake's body virtually attached to the back of her head, so when she moved her head from side to side it sent ripples down this virtual snake. We built a rig like a jungle gym surrounding Mowgli, and when she climbed around the rig the snake would sort of thread around after her.
How much of the actors' movements end up in the finished characters?
When it came to do the animation in post-production, the animators used the physical cues of what the actors were doing even if they weren't using one-to-one movements. So if Naomie Harris licked baby Mowgli, they would take that physical cue and animate off that.
Can performance capture be shot on a standard set? Or does it require a special setup?
It's a hybrid. We had our A-list cast for three weeks and we shot on sets that approximated the right eyelines for Mowgli, so Rohan Chand could act out every scene with all of his animal counterparts. Then we had live-action jungle sets built at Leavesden studios. We had another set of performance capture artists who emulated the A-list performances on set, standing in for the A-list actors. And then we went to South Africa and did the same process.
Does that leave room for you to change things during production?
We were able to keep it fluid right up until very recently, actually. The great thing about performance capture is that actors can come back in and we can rewrite lines and re-sculpt scenes.
Do actors have to learn new skills for performance capture?
When we sat down on the first day, the cast asked me, 'What's the secret of performance capture acting?' But I've always maintained that there's absolutely no secret. It's just building a character and attacking the role the same way you would if you were playing any live-action character. Acting is acting. You embody that character. You find the psychology and emotion for that character.
Of course, if you're playing a wolf or a tiger then you have to study them and observe their characteristics and behavior. But ultimately, you're humanizing them.
Part of the process, though, is when you see the avatar you're playing on a screen and, if you start to move, the avatar also moves. So the only form of training, I suppose, is learning to puppeteer that avatar. And if you want your character to have more of a hunched back, for instance, the engineers working with you can dial that in and have a slightly more arched back built straight into the avatar. So you would adapt and, as an actor, ... become one with the avatar.
What's the next big thing in performance capture technology?
The next stage is going to be is full facial real-time capture: very highly evolved, photorealistic retargeting of your facial expressions onto a character or a human face that's totally believable without any post-processing. That's really the holy grail at the moment. It's on the way. It's just not production-ready at the moment; it involves being stationary. Once you're untethered and once the information is caught on a head-mounted camera -- and is then real-time -- that's going to change everything. You'll be able to produce content straight out of the box. You'll film the actor and you will have content that's good enough to be screened, pretty much.
Was it fun to have a break from wearing the skintight performance capture leotard for your role as Ulysses Klaue in Black Panther?
I had enormous fun doing black panther. It was good fun trying to imagine what it would be like to have a folding cannon for an arm. That was quite tricky actually. It was quite technical. You have to remember the dimensions of it. Every time I lifted my arm up, I had to make sure that, by the time it had unfolded, there was enough room to get it all in the shot.
But I don't really draw a distinction between playing a character using performance capture or live action. To me, they're exactly the same.
Culture: Your hub for everything from film and television to music, comics, toys and sports.
Movie Magic: The secrets behind the scenes of your favorite films and filmmakers.