From his desk in the corner of the Ed Sullivan Theater graphics lab, Tim Luecke wields a power craved by Reince Priebus, Paul Ryan and countless other Republican poobahs: the ability to control Donald Trump.
Tapping at his keyboard and talking to his computer monitor's camera, Luecke, lead animator for "The Late Show with Stephen Colbert," operates a cantaloupe-colored cartoon avatar of the presidential candidate on his screen. He pulls Cartoon Trump's digital strings to make him grimace, wave around comically small hands and lip-sync Luecke's Trump impression in real time.
"It's all just just locker-room talk, Stephen," Cartoon Trump confides to a frozen image of Colbert seated next to him, Trump's eyes widening. "You know, that bus was so great, there's a locker room in it. OK. It was just me and Billy Bush in a shower in a locker room on a bus.
"It's not a big deal," Cartoon Trump concludes, as he furrows his brow and karate chops the issue aside.
Since March, "The Late Show," which is filmed at the Ed Sullivan Theater in Manhattan, has become a real-world training ground for one of the newest tools in TV storytelling. The late-night series has aired episodes that include live animations Luecke created of Trump, Hillary Clinton and a Bernie Sanders-inspired bird, allowing Colbert to interact in real time with these characters during shows. ("The Late Show" and CNET are both owned by CBS.)
New technologies to create live animations are gaining attention this year after Fox's "The Simpsons" and "The Late Show" became early adopters of Adobe's Character Animator. The program, which became available last summer, can shorten the time to create animations from weeks to hours by allowing animation puppets to immediately mimic the movements of someone's lips, shoulders and eyes using a face-tracking camera.
Adobe isn't alone in creating different flavors of motion-capture technology and real-time digital animation, but it is one of the first companies to offer up a relatively cheap and easy-to-use version that can be operated live. With the help of this software, Cartoon Trump may be just the beginning of live animation. Perhaps Saturday morning critters will become more interactive or Tony the Tiger will do a Facebook Live chat (admittedly, that last one sounds horrible).
"When somebody wants their cartoon personality to communicate to society [about the day's news], well, you don't have time to do regular production. That's why it doesn't happen," David Simons, a long-time Adobe executive and one of the brains behind Character Animator, said. "I think we will see more of that."
From Crunchy and beyond
For "The Late Show" team, the forebear of their live animation work was a character called Crunchy the Squirrel. Crunchy, who aired in a skit last December, was premade, so the squirrel, its lines and all its movements were created before the show was taped. (Premade and post-production animations have existed for decades. Colbert's predecessor, David Letterman, once interviewing Beavis and Butthead on the show.)
During my visit to the Ed Sullivan Theater last week, Luecke and writer and senior digital producer Rob Dubbin told me they were happy with the Crunchy skit. But the animation work took about a month, and with each step of the process, there was less and less room to make changes.
"There was no way around prerendering the animated side of the conversation," Dubbin said in his office, which is decorated with a "Millennials Love Trump" sign. "No matter what happened, you were always going to be going back and forth between a human performer and a pretape."
Colbert butted up against a similar limitation during one of his last shows on "The Colbert Report" on Comedy Central, when he chatted up a beautifully animated but still pretaped Smaug from the "Hobbit" movies.
A few months ago, Luecke started toying around with Character Animator. Dubbin jumped in to figure out how they could use the software on the show to produce animations faster and let them improvise in front of a live audience.
They first tested a cartoon Irish ancestor of Colbert's for St. Patrick's Day, though that character never made it to air. Then, in late March, the show produced its first episode using Character Animator, bringing to life a pro-Sanders blue bird who complained about the unequal distribution of bird seed. Two days later, Cartoon Trump aired for the first time. While Colbert spoke to his cartoon guests, which he could see on a nearby monitor, folks behind the scenes made the character talk and move from a puppet performance room offstage.
"The Simpsons" got in the mix in May, with Homer answering viewers' questions on the fly during the show's first-ever live segment. In July, Colbert interviewed Cartoon Clinton during a live Democratic National Convention show (most shows are taped earlier in the day) as the Clinton avatar sat beside Colbert onstage, stopped to thank the audience for cheering and played the harmonica.
Cartoon Trump and Cartoon Clinton have since become regular guests, with a September episode including Clinton coughing up a cat while Colbert sat nearby in shock and her (temporarily) dying mid-interview as she answered questions about her health. Colbert wasn't available to be interviewed for this story.
Neither campaign responded to requests for comment on Trump's and Clinton's cartoon doppelgangers.
Since the bird skit, Luecke and Dubbin have worked more closely with Adobe, with the software company answering requests for new features in Character Animator. Luecke wants his puppets to blink on their own, instead of requiring him to repeatedly hit a button on his keyboard. Adobe soon after turned on an "auto-blink" function.
The day after the election
In the hypercompetitive world of late-night TV, "The Late Show" has struggled in its first year under Colbert to match the ratings of "The Tonight Show" and, at times, "Jimmy Kimmel Live." But its live broadcasts and cartoons have helped give the show its own feel and allowed Colbert to weigh in on a wild election season.
The question is whether other shows may start using similar animation tools. Some lower-profile productions on YouTube are already experimenting with Character Animator. Simons, of Adobe, said his company has been contacted about the software by several interested parties, including some he couldn't yet mention.
"I'm happy to have them experiment and make some crazy failures to come up with a completely new way of communicating," Simons said of people using the software.
He added that he sees the potential for traditional animators to offer their characters for sale in a future digital puppet marketplace. Those characters could then be used in TV shows and Snapchat filters or as internet avatars.
For Luecke and Dubbin, the next test for the technology may come very soon; the show's next live broadcast is Wednesday night, right after the final presidential debate. Beyond the election, though, they see a lot of possibilities for how to use the software.
"As cool and as new as this is, I still equate it to making a puppet. Something that's been around for a while is creating a character that someone can interact with," Luecke said. "The sky's the limit. We can start developing a cast of characters that are cartoons that Stephen interacts with."