Nicki Minaj SpaceX's historic all-civilian launch: How to watch Apple event: Everything announced Rivian R1T electric pickup West Side Story trailer Woolly mammoths are coming back
Exclusive

Holograms for your phone: How second screens could share 3D AR in 2021

Ikin's tech promises a glasses-free holographic display as a phone accessory this year, with holographic chat on the table. Here's our exclusive video chat and a brief video preview.

wasp-image.png

Ikin's promised holographic tech could show glasses-free effects from your phone with an added accessory.

Ikin
This story is part of CES, where our editors will bring you the latest news and the hottest gadgets of the entirely virtual CES 2021.

The future of holographic augmented reality seems like it's on glasses, but it's also on phones. One company is planning an in-between solution this year that sounds bizarre, but also is not far off from what already exists. Ikin is a company promising a holographic glasses-free technology, called Ryz, that will connect to phones and offer a way to share 3D images with others. 

While AR headsets are currently few and far between, phones are clearly where AR is at for most people for the time being. Ikin aims to build its peripheral for current phones, turning them into half-holographic devices that make me think of Westworld's future tablets.

Now playing: Watch this: Holograms for your phone: how one company plans to do...
14:07

I saw the news and was instantly extremely skeptical and curious. So I talked to Ikin's founders over Zoom. They also showed me brief videos of the tech in action (according to Ikin) which you can see for yourself. The videos I saw were prerecorded, but what I didn't get was any live demo, or any peek at what the promised device actually looks like. What's more interesting to me is that its tech, surprisingly, is set to emerge in beta form by the middle of this year.

My feelings, seeing Ikin's concepts, are similar to when I was visiting Magic Leap for the first time. I'm wondering how this will actually feel in person, versus the "hologram in your hand" idea that's being presented.

Ikin's founder and CTO Taylor Scott and CEO Joe Ward answered some of my questions, at least. The company's phone accessory will combine a display and lens and possibly its own co-processor, displaying glasses-free light field images that can even be seen in bright daylight, according to the company. I've seen glasses-free displays before: Looking Glass Factory's displays look like glass boxes where glowing 3D objects and videos can hover, and be seen from a limited number of angles. Sony has a light field display for professionals that I haven't demoed. 

"The library and universe of content we're about to have in the next five years is going to make old 2D photos look so incredibly archaic," Scott says of the potential for the company's first product.

There have been promises for game-changing 3D tech before, though -- need I remind anyone of the Red Hydrogen One? But Ikin's concept is definitely weirder. The holographic display sounds like it floats, or creates the impression of floating. The 3D objects could be pulled from a phone's existing library of assets. I asked about the growing number of 3D scanning apps and Apple's new lidar-equipped iPhones. Ikin's tech looks to work with those and will use the ARCore and ARKit capabilities of phones to help project its images into the real world. I'm just not sure how, though.

The holographic images will be able to be controlled from some sort of finger control (or hand-tracking), with an ability to "feel" feedback. Ikin didn't confirm whether it would be using anything similar to the only in-air feedback for gestures I've ever tried (Ultraleap's ultrasonic haptics), or something else. 

One video I saw, which you can also see above, showed an insect hovering near a hand. It's hard to tell what's holographic here, versus just superimposed. But Ikin says the 3D holographic images will recognize surfaces and be capable of occlusion (hiding behind real objects), much like AR 3D objects can already achieve on Android and on Apple's latest phones. The resolution doesn't look impressive, but Ikin promises a resolution similar to what a phone can display, one that they claim looks much better than what the videos show off (admittedly, I've found, AR and VR captured in 2D videos usually don't end up looking very good).

A second video shows how a phone's display could extend into the holographic second screen: a Breakout-like game has a ball shooting off the screen and into a floating wall of bricks. What apps would support this second screen, though? It sounds, much like most mobile accessories and add-ons, like a chicken-and-egg challenge. Ikin wants to incorporate other apps, but Android and especially iOS aren't currently all that friendly to meshing AR headsets and accessories with their OSes. 

But Ikin absolutely envisions dual-screen ways that the holographic and flat phone screen you already have will interact. "You can have your streaming service played on the 2D phone, and you can have all the chat features, your text messaging systems, emailing systems, all within the hologram," Scott says. "Nobody's ever been able to really type in a holographic medium before."

More interesting to me is the company's promise of live holographic chat, where you could conceivably have a conversation with a friend who floats in space, able to be seen with up to four people in the same room at once. No AR system I've seen has been able to combine live camera feeds with AR chat yet. Companies like Spatial are experimenting with ideas, but those fusions aren't able yet to use front-facing video recording while AR is being engaged at the same time. Again, I'd like to see how this would even work in real life.

"We don't impact the battery," Scott promises of the Ryz accessory's impact on phones that will work for it. "More specifically, we can reach a variety of phones with one module."

Ikin plans Ryz for Android phones first, then iOS. An early version of the device should arrive by midyear. But Ikin plans on developing its tech beyond just phones, to computers or even standalone devices. According to the company, the tech will also be able to dovetail with smartglasses. It all sounds like a tall order, but also, is it really that far-fetched compared to tech I've already seen? I'll be curious to try it out when I can later this year.