Spatial Personas Have Turned My Apple Vision Pro Into a Telepresence Machine

After spending an hour and a half at night with a friend, drawing, playing games, watching movies and looking at models, I realized I forgot to say goodnight to my kid. It was after 10 p.m. In my office, I felt like I was hanging out with someone for ages in all sorts of worlds. I was alone, though. Norman Chan, co-founder of Tested, was miles away in a hotel room. We were just teleporting into each other's spaces, and for a while it felt like we were really sharing time together.

Apple made a small but very important change to the way its Personas, its word for avatars, look and interact on the Vision Pro headset. Spatial Personas, a feature that's now available to test in developer beta mode, frees Apple's ghostly scanned likenesses of friends and colleagues from their windowed confines, allowing them to seem to like they're floating directly in your space, pointing and even interacting with the things you're using in mixed reality. It's a feature Apple promised back at WWDC 2023 when the Vision Pro was announced, and it's suddenly here, over a month after the Vision Pro launch, two months before Apple is expected to announce even more software updates at WWDC 2024.

I've dreamed of mixed reality telepresence before. Demoed it, even. I tried AR glasses experiences with startup company Spatial where I collaborated with virtual avatars floating in the world around me back in early 2020. Microsoft's Alex Kipman appeared as an avatar in my home office while I wore a HoloLens 2, testing Microsoft Mesh in 2021. Meta's had virtual spaces and metaverse apps where I've tried theater, work and games. And Google showed me light field displays where real people, projected as 3D versions of themselves in front of me, made for realistic telepresence chats in 2022.

Apple's floating 3D Spatial Personas feel like a blend of all of these, and the way that Apple's Personas emote and animate are even more compelling when allowed to be active and more present. Spatial Personas activate via a small button toggle on FaceTime calls in Vision Pro that looks like a 3D head emoji. Whoever's chatting with you just drops in as a head, part of a set of shoulders and floating hands. It's not a full body and the backs of people's heads are still empty. It's like you're chatting with the half-scanned version of them, like the holograms Ericsson showed CNET's Katie Collins last month. But it's compelling, and feels eerily real.

Apps that allow SharePlay can be activated to share a session between Personas, with up to five being able to join at once. The entire shared app space becomes a bordered zone in 3D space that can be dragged around and repositioned in your room. The shared items or screens move, and so do the Personas, but you can stand still. Your relative position to the other people is then shifted accordingly.

Apple's collaborative Freeform app, a sort of virtual whiteboard, was never that interesting at the Vision Pro's launch. Now, it's fascinating. Norm and I drew in real time on the whiteboard, making me wonder what sorts of creative brainstorming sessions or collaborative art or modeling apps could emerge next. There didn't seem to be any lag.

Movies are an unexpected blast. I tried SharePlay viewing movies in the Vision Pro before and it didn't feel very shared at all. Now, Personas are dropped right next to me in Apple's virtual cinema space. The Personas look more ghostly than I'd prefer, and everyone seems to need to be placed at the same agreed-upon distance to the screen, but it made it feel like virtual movie night could be a very real experience. Apps like Bigscreen on the Meta Quest have features like this too, but Apple's multitasking in VisionOS makes it possible to watch movies like this and also open other apps at the same time. It's uncanny.

We also played a chess game, which was fun. I played Game Room before with friends over FaceTime with Personas in bordered windows. Now, our hands can reach out and grab chess pieces. I could get up, walk around the table and consider my move.

Norman also dropped some of his own 3D models into my office, which we marveled at. Stuff from Star Trek and The Mandalorian: It felt like we were kids sharing toys.

Time sort of melted away, and so did my sense of reality and space. For a little while, it felt like a friend came over to my cluttered upstairs office space. And then, he was gone. What can come next with this? My mind races to thoughts of collaboration, theater, art, improvisation. It's the vision of telepresence that Meta's Mark Zuckerberg has preached for years. But it's becoming very real on the Vision Pro right now… except, there just aren't that many people who even own one. For those who do, it's suddenly become a lot more compelling as a real collaboration tool.