Neon's CEO explains artificial humans to me and I'm more confused than ever

Tall displays flank a stage in the Las Vegas Convention Center's Central Hall. People of all ages, ethnicities and genders smile from the screens. The videos look like streams of real people. But they aren't actually humans. They're Neons. 

One of the buzziest companies at CES 2020, also called Neon, debuted late Monday at CES 2020 here in Las Vegas. The mysterious company, emerging from the Samsung Technology and Advanced Research Labs (aka STAR Labs), described its technology as "a computationally created virtual being that looks and behaves like a real human, with the ability to show emotions and intelligence."

Basically, Neon makes video chatbots that look and act like real people. Neons aren't all-knowing smart assistants, androids, surrogates or copies of real humans. Instead, they're designed to have conversations and behave like real humans. They form memories and learn new skills but don't have a physical embodiment, at least not now. Neons can help with "goal-oriented tasks or can be personalized to assist in tasks that require human touch." They can act as teachers, financial advisers, health care providers, concierges, actors, spokespeople or TV anchors. 

So yes, we're slowly sliding into Black Mirror territory.

Watch this: Meet Samsung's spooky robot cook and cute robot assistant

03:04

The Neon artificial humans are powered by two pieces of its proprietary technology. The first, called Core R3, stands for "reality, realtime and responsive." That's what makes the Neons respond quickly and in a lifelike manner. The second is called Spectra, which is responsible for intelligence, learning, emotions and memory. 

You'll be forgiven if you're kind of confused by all of this. I am too, so I jump at the chance for the first media interview with CEO Pranav Mistry on Tuesday afternoon at Neon's CES booth. In a rapid-fire half hour, we cover everything from how Neons will respond to abusive language to the moral responsibilities for creating what Mistry calls a "new species." 

But first, a basic question. What, exactly, are Neons?

"They're virtual beings that look and behave like us," Mistry says from a small meeting room in his company's booth. "That gives us the nexus between our world and the digital world. You can clearly say they're not an AI assistant."

If you're feeling confused, you're not alone. 

Making friends

Mistry tries to break it down a little more. A Neon is a "virtual individual" who can "go anywhere we see a human can go." 

You'll be able to have video phone chats with them or display them on your smart TVs, just like you'd talk to your parents using Skype or FaceTime. They won't be miniature but will have a "lifelike quality and lifelike scale," meaning during a video chat on your phone, you'll see them as if they're a regular person you're talking to. 

"In the future, holographic can come and be a part of it," Mistry said. But don't expect them to show up as physical robots, he said. That's not the plan for Neon. 

While they can borrow traits of real people and have a similar look and voice, they can't be exact copies of existing humans, a potential deterrent against the concerns that this is just a next-level deepfake. And each Neon is unique, with its own personality. 

"The first phases of Neons are real people's likenesses," Mistry said. "But how they move, how they behave, how they learn ... is created by Core R3."

The technology has the ability to generate a completely original look of a person, but we won't be able to design our own Neons. 

"When you meet a friend, you build that friendship, not that you build that friend," Mistry said. In the same way, businesses who hire Neons can't decide what they look like, he said. 

"If a bank says, 'I want to hire a representative who can speak Spanish,' the bank will not be able to select, 'I want this kind of look,'" Mistry said. "When you hire a person, you hire a person."

Digital 'dear diary'

Mistry envisions Neons to be sort of a digital version of a "dear diary" or a close confidant. 

They're run by advanced artificial intelligence, and they're constantly learning and building memories from their interactions with you. They learn to become more humanlike, which is shared with Neon's central Core R3 system. But those specific experiences they had with you stay private and encrypted. Only you and the Neon you interacted with know what happened. 

"They're keeping track of personal references and learning from you, but that's not the data Neon is evolving on," Mistry says. "They're learning from the interaction with how humans talk, how they behave and smile. ... That particular information is passed to the Core R3 to build a much better version of Core R3. But every interaction between you and Neon is locked."

Like a real, human friend, the memories of interactions are tied to one specific Neon. If you interact with a new Neon, it won't be able to pick up where you left off with the previous one. You're starting over, as if it's a new human that you're meeting.  

Neons aren't all-knowing assistants like Amazon's Alexa, Mistry said. They can pull information from the internet, like how a regular person accesses the web. 

But he envisions Neons being skilled in different areas. For instance, one might be an expert in yoga, so you subscribe to that particular Neon to teach you poses in your living room for an hour each day. Another could be fluent in Spanish, helping you communicate with people you meet while visiting Madrid. 

"Every Neon doesn't know everything," he said. "They learn the way people learn. They're not perfect because humans are not perfect. That is what makes them human."

Neons are not for sale, Mistry said. You instead subscribe to them, and businesses will be able to hire Neons to do things like speak Tagalog to bank visitors.  Mistry also won't be licensing or selling the Core R3 technology that makes the Neons respond quickly and in a lifelike manner.

If you're mean to a Neon, it will respond the way a normal person would. The company isn't designing Neons to be eternally patient and gentle. "If you make a Neon upset or angry, we want you to make up in the same way," Mistry said. "It takes time."

Neon is still in its early days. While one important part of the technology is working, another, Spectra, needs more development. Spectra is the part that gives the Neons emotions.

Now we need to see them for ourselves. Tune back to CNET for more coverage of Neon, including impressions from demos.