So... this is my experiment. Just knocked a controller onto the floor because it was invisible. But my keyboard's not. It's here in front of me. But my hands are ghosts. I see them float over the keys. It looks like I'm the one who's virtual.
What's happening here?
That's the first thing I typed when I paired a Bluetooth keyboard to theand pointed the browser to Google Docs. Thanks to a , typing and seeing your hands hovering over a virtual keyboard recreation is possible now. Experimental, but possible.
I also wrote the below:
I am seeing the keys and the screen, and now it feels like I'm really in a room somewhere. One with glowing spheres, where a screen hovers, and my keyboard. The Oculus Quest 2 is now somehow kind of a computer for me.
I try standing, keyboard on a bookshelf. The angles are wrong, the monitor too high. Then I go to my desk, lay the keyboard down there. The browser window hovers a few feet away, and my keyboard feels more lined up. I can see and type and get a sense of my hands and keys in the periphery, which helps me line up keys by touch.
My hands appear as this bit of cut-out passthrough, overlaid on the keyboard image which is mapped to where this particular physical keyboard is. I can see what keys I type... they glow blue.
And then, when I screen record, my hand outlines disappear. I guess in that sense I can't really record myself typing for you as I feel it, but you may get a sense of it anyway.
And now my hands won't come back.
Here they are again.
To try this VR trick yourself, you'll need Oculus Quest, which is rolling out to headsets now. You also need a particular keyboard: the . The K830 has Bluetooth, and its own trackpad and buttons. Go to the Oculus Quest's settings and make sure hand tracking is on, then you can pair the keyboard and turn on keyboard tracking in Experimental Features.
Not everything worked for me at first: I restarted the headset and then reset all the experimental settings to default. Then tried again, and it kicked in. (Some people on Reddit suggest a factory reset is needed, but I didn't go that far).
You can pair any Bluetooth keyboard and mouse with an Oculus Quest 2, but the K830 has the extra bonus of being visible in VR, and triggering the hand-passthrough effect.
Here's how you set it up:
1. In the Oculus Quest settings, go to the Experimental Features section and find the Bluetooth pairing option. Tap the pairing button to connect the keyboard. Make sure hand tracking is also on -- that's under general settings.
2. Once this is done, you can try selecting the track keyboard setting in Experimental Features to adjust the opacity of your ghostly hands. (This adjusts how much you can see through them to the keyboard.) But that didn't work for me, so I ignored this step.
3. You should see the K830 keyboard popping up, like a graphic rendering of it, on top of the Oculus main menu.
4. If you don't see it, try resetting Experimental Features settings to default, which is what worked for me. Make sure your keyboard is turned on. If that doesn't work, reboot the Quest 2.
The keyboard needs to sit still to be tracked properly, so you'll need to set it on a flat desk. Also, Oculus suggests the room lighting shouldn't be too dark. When I put my hands near the keyboard, I saw my ghostly hands and fingers appear over it. Weird, but interesting.
At first it was offputting, and I didn't have the same sense of key awareness that I normally do at a laptop (or even a phone or tablet). The craziest thing for me was that when I relaxed a bit, I felt like I was at a computer. My peripheral vision's awareness of my hands and the keyboard was enough to help my typing stay on track.
There are a lot of caveats here: I used the Oculus browser and typed into Google Docs. Even then, I got a warning telling me Facebook's browser is not a supported browser for Google apps. It seemed to work fine, but be aware.
Other apps didn't seem to work with it yet. Immersed, an app that can extend my Mac to the Quest 2 and use the headset to cast multiple virtual monitors, didn't recognize the floating keyboard or my ghostly hands (it just let me type invisibly, which you can do with any keyboard). The other VR browser on Oculus, Firefox Reality, didn't seem to acknowledge the new passthrough typing effect either.
Could Facebook's evolution of typing in VR make the Quest 2 into a Chromebook? A travel computer? What about when Facebook tries to get this typing trick to work with no keyboard at all? I think about how my typing, combined with my hand overlays, could be helping Facebook train that future algorithm. My typing speed increased, the more I used it. It started to feel natural(ish).
But for now, I'm not giving up my laptop or my tablets or phones. The Oculus Quest 2 is dipping its toe into work tools, but the road ahead may be bumpy and experimental.