Prop gun kills one on set of Alec Baldwin movie iPod at 20: Inventor looks back Moderna booster approved Ryan Gosling could play Ken in upcoming Barbie movie Uncharted movie trailer PS5 restock tracker

Facebook is developing a way to type using VR on tabletops

Hand tracking and VR, typing away with no keyboard

frl-vr-typing-screenshot.png

Facebook Reality Labs' VR typing research is working, but with specialized marker-gloves for now. Eventually it could be on headsets.

Facebook

Doing office work in VR isn't easy. It isn't just the lack of software plugins for VR and most productivity apps, since VR headsets can already transform into wearable monitors. The keyboard part is what gets really frustrating. Facebook's working on a way to bypass physical keyboards altogether with hand tracking.

Facebook Reality Labs' latest research project, being presented at the Symposium on User Interface and Software Technology, demonstrates hand-typing on a desk at up to 73 words per minute. The project isn't ready for Oculus Quest VR headsets yet, which already have hand tracking but lack the support for typing input. The research project looks to be using extra tracking sensors right now, along with specialized gloves studded with motion-capture markers. But the idea is to eventually evolve the Oculus VR hand tracking to enable typing, just like it does hand tracking now.

frl-keystroke-motion.png

Facebook's motion model sees the keystrokes as they're happening in 3D space.

Facebook

"To support touch typing without a physical keyboard -- and without the benefit of haptic feedback from individual keys -- the team had to make sense of erratic typing patterns," Facebook's post says of the research project. Facebook's typing AI is adapted from speech recognition, but searching for particular keystroke motions amid the general finger movement. It works predictively to anticipate letters as they're being typed. The research so far is producing typing at "73 words per minute with a 2.4% uncorrected error rate using their hands, a flat surface, and nothing else," Facebook's post claims.

Facebook's already supporting certain VR-optimized physical keyboards with Oculus Quest VR by the end of the year, but keyboard-free typing could be a way for VR and eventually smart glasses to work without any accessories at all.