Speaker 1: I know this is hard for some of you to hear, but nobody really cares about virtual reality, at least not in the way folks care about our other gadgets. VR is not a big seller. So how in the world is Apple going to get anyone in the real world to care about its reported $3,000 face goggles coming soon. How is Apple actually changing the conversation to make this a desired thing and not just nerd tech for your face? Well, there is [00:00:30] something Apple reportedly has one more thing in its headset that no one is pulling off yet recreating your entire face and body movement in a VR, FaceTime chat. So it's not a cartoon you, it's virtually real looking. You, you, this is either going to be pure magic or pure terror, and I can't wait to dive into all these new VR report details with you.
Speaker 1: I'm Bridget Carey, and this is one more thing. Bloomberg's Mark Goman is dropping new detailed nuggets about Apple's [00:01:00] vr, ar mixed reality headset, which he reports will likely be named Reality Pro. It's said to be revealed soon, maybe even this spring, and it can toggle back and forth between augmented reality and virtual reality. That's why folks call it mixed reality. You've heard us banter about this headset since we broke the news of its development in 2018, but Germond often gets a lot of his apple scoops, right? And he is out here squeezing more details out of the headset practically every week. [00:01:30] And when going through all the details of how he says the headset is going to work, I'm thinking this has been done before. The features don't seem all that surprising. That is if you follow what VR headsets can already do. For example, the report says with Apple's headset, you'll not need to hold a controller.
Speaker 1: It has voice commands, but it also maps your hands and uses cameras inside to track eyes to know where you're looking. So if you look at something, you just stick your hand out, pinch [00:02:00] your thumb and your index finger together and do a little clickety click. Well, I'm sure Microsoft would like some credit because the holo lens already got that trick down a while ago. The whole finger pinching for your mouse thing. I mean, and this also is true in the latest meta quests too, and Quests Pro, these come with controllers, but it also lets you select things with your hands and it knows where you're looking inside the device. So not a game changer feature yet, but okay, let's keep going. [00:02:30] There's also a report about a digital crown, maybe designed similar to our little Apple Watch knob, where you can switch between augmented reality and virtual view by doing a little twist on the headset.
Speaker 1: So essentially you can see the room around you with screens and apps just floating around, or you can choose to get lost in another world and not see the room around you. That's already easy to do with the MedQuest Pro, which you can see here from what my colleague Justin Tech recorded inside his headset. [00:03:00] Instead of a digital crown, he just double taps the side of his headset to switch modes. Now you're in vr, now you're not. Now you're in the room. Now you're not and not a new thing to wow us yet Apple. Let's talk about navigating applications in vr. The report says Apple is gonna let you do your work on this virtual workstation, so you can see your Mac displayed in vr, but control it with a physical mouse, an actual keyboard, an actual track pad. And [00:03:30] this interface is also supposed to look very familiar like iOS, but the apps could just be floating around you.
Speaker 1: Ooh, mail in vr. Exciting. This is done. Now, magical Leap had a virtual office space that blended real with the fake. And let's also look at what you can already do with a Quest Pro. In this example, the headset can map the keyboard. Exactly. It's just using an app to connect it all. Just in here shows how he can see his real hands [00:04:00] on the keys, but if he moves away, the hands go back to cartoony hands. And now we are walking right into the zone where Apple could actually wow us all because it's said that the cameras on the device can scan your body to make a real image of you for someone else to see instead of making you into a floating cartoon. First, I should explain how this kind of stuff works on the Facebooky side. In Horizon Worlds, you make a cartoon [00:04:30] version of yourself and the Quest Pro can use its cameras to tell when you're making a facial expression and move your mouth to match.
Speaker 1: But what Apple is doing could go much further. Bloomberg's report says the Apple headset cameras will map your whole face and your full body realistically to render an image of you in vr so you can really feel like you are in the same room with someone. But it could only be done on a one-on-one FaceTime video chat when both folks are wearing the Apple headset. Okay, so [00:05:00] if this is true, apple has apparently achieved ultimate camera wizardry, like actual magic. Someone call up David Copperfield because they're capturing things when it's not pointed directly at the things. So the cameras are processing the different angles to make me look like Bridget or virtually Bridget. Apple has been perfecting its camera smarts for ages. Now, let's not kid ourselves. Apple is a camera company. I guess I shouldn't be surprised, and this makes me think of a new iPhone webcam trick that Apple [00:05:30] recently launched.
Speaker 1: It's called Desk View. Apple may be using something similar for this realistic body imaging. In this example video, an iPhone is being used for webcam FaceTime call, and there is an option to show someone what's below at your desk without moving the camera at all. Now, sometimes the image can be a little warped, but the computer is trying to compensate the image it sees in a wide angle and make it look like a top-down shot and the camera is not pointed down. So [00:06:00] ideally, they could just point all these cameras in the headset to my chin and my shoulders and just map out a non warped version of my body. Now, obviously this is a lot of processing going on and it's only working for two people. If the FaceTime call is more than two people, the magic is gone. It's said to go back to emojis for everyone else on the call.
Speaker 1: No different than what Medic can do, but if Apple can really pull this off, really make someone look like themselves in vr, this could be the thing that has people losing [00:06:30] their minds and running to try Apple's version. Then again, it could vary easily go into uncanny valley territory. We can detect the slightest things that are off in fake avatars. I don't wanna pay three grand to have Apple turn me into Polar Express, Bridget, you know what I'm saying? To make sure I'm not missing something. I need to talk to the guy who has tested almost every VR system known to man Cena editor at large. Scott Stein. Hello Scott. I need to know, has anyone pulled off real humans inside VR before?
Speaker 2: [00:07:00] Not really. And that's one of the things I'm most curious about with these reports. I mean, I think the closest that we've seen is Meta doing some advanced avatar experimental work called Kodak Avatars that I saw at their reality research lab in Redmond last year. This looks like a kind of a hyperreal conversation, almost like video game meets reality. I got to have a a a brief example of this, um, at their, at their research lab, and it felt like I was, you know, talking very close [00:07:30] with somebody in a much more realistic format. Everything else that I've seen for the most part has been much more paired down like cartoons in vr. But there are companies that are looking at Photoreal avatars already. You see a lot of them, uh, being talked about through various AI programs. Also, unreal already has metahuman, which is a, a way to build really realistic avatars, but, but not yet for any VR purposes that I've seen.
Speaker 1: I feel like if Apple pulls [00:08:00] this off, it could be Apple's big wow twist here and I don't wanna like walk ahead of myself, but do you think they can actually figure out legs in VR
Speaker 2: <laugh>? Well, not just legs, but again, I don't really know how the the face thing would work because you know, the challenge with VR is that you're wearing the headset and so people have to have stand-ins. Does this mean that Apple would do some sort of scan of your face? Would there be cameras that could pick up cues? Would it be doing facial tracking or like you said, body tracking right now, the Meta Quests Pro that does some [00:08:30] face and lip tracking in addition to eye tracking and, uh, there are body trackers that some VR headsets are compatible with, but we haven't seen anything that does the kind of like full body awareness through cameras built in that this report said might be
Speaker 1: Possible. All right, so you also read this Bloomberg piece. What else in the reported features has you intrigued?
Speaker 2: Well, a lot of the stuff that we see in this report, I feel like I've seen in other headsets. You know, it's kind of like this, um, conglomeration of [00:09:00] things, you know, working with a, a PC or a Mac, uh, that's stuff that a lot of VR headsets can do to create monitors. The battery pack reminds me of the Magic Leap. You've got hand tracking only it seems, which is something that like HoloLens two two and, and Quest two, uh, can do hand tracking. So I, you know, I feel like I see all these pieces, but I'm curious about the promise that it will just have hand tracking whether or not Apple will have some sort of controller to go with it because hand tracking and eye tracking, you know, aren't [00:09:30] as accurate or as reliable for everyday tasks in a lot of things that I've used compared to what, you know, this seems like it's shooting for.
Speaker 2: Uh, would it also work with computers and phones? It sounds like it, but you know, how seamless would that be? I'm also really curious about prescription glasses because I wear prescription glasses and I've seen all the prescription insert efforts so far seem to ignore my ex extreme, uh, prescription needs, um, and are hard to get fitted for. Would Apple make that easier? Would they turn the Apple [00:10:00] stores into optical shops, you know, sort of like a Warby Parker and be able to, to get your prescription, um, and make that a simpler service? So I I'm also really curious about that,
Speaker 1: Scott. That would be something else. I mean, apple stores are already in every mall. Now you can get your prescription figured out an Apple store too. That would be far out
Speaker 2: <laugh>. That's the road to ar glasses. I mean, I'm also curious about how much of an everyday device this is, you know, like it's $3,000 supposedly. Um, you know, is this something that, you know, really will be, will, [00:10:30] will be pitched as a, an everyday device or something more for creators? Sort of like some really high-end, uh, VR rigs that I've seen that connect to PCs. So that positioning is, uh, I'm really curious about too.
Speaker 1: Thank you, Scott. I'll leave you to go back to your virtual worlds
Speaker 2: <laugh>. Thank you.
Speaker 1: It kind of seems obvious, but Apple is really leaning into its media connections for content. The report says Apple's gonna make content with Dolby Disney turning some of its own Apple TV plus content [00:11:00] to work in virtual reality. Now, of course, apple has a lot of big connections already with publishers of movies, music and games, but I'm still not sure if that is going to be the killer feature. Who wants to watch video in a fake VR theater? But could Apple Music make deals to have a special concert in vr, extra sports content? Maybe. I suppose if they get creative, the content can be an extra incentive, but I want to better understand what is important to you in a VR headset. What [00:11:30] in these reports has you intrigued? What magic will Apple need to pull off to make you care about using vr? Or maybe it's not a trick and it all comes down to games, or maybe just the mere fact Apple is doing it is why you're gonna care. Be honest with yourself here. Drop your thoughts down in the comments. I'm Bridget Carey, thanks for watching. And does anyone think that health tracking is gonna be the hot feature? I'm not sure I'm ready for virtual reality fitness plus workouts. Good gravy.