Meta wants you to feel the metaverse with high-tech haptic gloves
Meta's head of research discusses the latest prototypes and why feeling is the next frontier.
- Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
A haptic glove research prototype from Reality Labs Research.
VR visuals can only look so good, given current display technology, and we may already be approaching their limit. But what about our other senses? Facebook parent Meta's dream of an immersive metaverse makes big promises, but current VR interactivity is limited. The Oculus Quest's controllers, for example, still feel more appropriate for gaming than work. After immersive video, mixed reality, face tracking and spatial audio, haptic feedback could be the next frontier. Details of Meta's latest prototype hardware show where the research is heading.
The idea of slip-on vibrating gloves that let you feel the virtual world has been part of our sci-fi vision of VR for a long time. Meta's research arm has been working on haptic gloves for seven years, and even now the gloves that have been developed aren't yet portable. But they could be someday.
I didn't get a chance to try Meta's prototype myself, but Sean Keller, the company's Reality Labs research director, and Michael Abrash, its chief scientist, talked me through recent developments. They explained why haptics are so important to Meta and where this all fits into the future of AR smart glasses.
Gloves made of... bladders?
Meta's prototype inflatable-bladder gloves are a step towards what could eventually become a pair of consumer gloves. A new look at the tech, though, shows that it'll be some time before we see them bundled with a Quest VR headset successor.
The latest glove designs use microfluidics to push air through a series of bladders across the gloves, which is less conceptually challenging than filling a pair of gloves with lots of little motors. It's an odd idea, like Meta's version of a Dune stillsuit for your hands.
"Literally, we're changing the stiffness of the material," says Keller. "We have ones that use air to move something across your fingertip up and down, or laterally and inward -- that helps you create that shearing force -- and ones that are little bladders that create pressure."
Miniaturizing pneumatic actuators (pictured), which use air pressure to create force.
But Meta's confident that the haptics can do things at a level that hasn't been possible before. Facebook Reality Labs Research tried a demo in 2017 that created the sensation of dropping balls made of different materials on your outstretched fingers. The wooden ball felt different than the marble ball and the squishy foam ball. The illusion is a combination of hundreds of pressure-applying bladders (pneumatic actuators), and the visual inputs from VR. According to Michael Abrash, it just wouldn't work the same way on a 2D screen, making this tech specific to AR and VR.
This tech exists in a lab now, but Meta's research indicates a need for a whole haptics rendering engine. It's reminiscent of how the PlayStation 5's DualSense controller has needed to develop new ways of creating immersive haptics, but on a larger and more complex scale.
"I've got all these bladders and things on my hands, they've got tubes. And the next one, I'm going to have 10 times more. It becomes a huge problem from a systems perspective, you can't solve it without someday building something like microfluidic microprocessors," says Keller.
Meta is working on ways of creating the right materials for durability, too, so the gloves could actually be worn somewhere. But for right now, the gloves are purely in research mode.
A 2017 experiment used a fingertip to device provide haptic feedback as a series of virtual spheres made from different materials — wood, marble, foam — fell from the sky in VR.
Touch as a way of conjuring tools
Based on Meta's demonstrated demo concepts, a lot of the uses for haptic gloves right now look like simulations of hand-based activities: playing Jenga, thumb-wrestling or picking up objects. A lot of these ideas already are sort of possible with hand tracking, just minus the physical feedback.
Where Michael Abrash sees the tech making a big difference is in simulating virtual tools. Typing on a keyboard, or holding a sculpting tool or a brush that isn't actually there, could mean that haptics finally make the idea of working in VR less awkward. Of course, for now, that would also mean wearing big, strange gloves that are tethered with cables and tubes, which is equally awkward.
"The question becomes, how well can the gloves emulate the tools? And I don't know the answer to that yet," Abrash says. "A virtual keyboard could move with your hands, so as you're typing, if you drift around a little, we can still infer what you were meaning to type."
An earlier haptic glove research prototype.
Smart glasses won't use haptic gloves (not now, at least)
Abrash doesn't expect to see haptic gloves being used all the time with smart glasses; instead, the company's been working on electromyography. EMG uses wrist-based bands that can sense motor neuron signals, turning them into gestures and controls. Meta's already laid out a roadmap for where that tech might be going. Those bands will use wrist vibrations for feedback, but Meta's full haptic glove research could dovetail and lead to products that are a fusion of both.
Abrash sees the haptic gloves and neural input as being a feedback loop that could make invisible tools work. But that combination of tech is still theoretical: right now, Meta's just closing in on making the haptics feel convincing.
"One gap with EMG is that it's unidirectional, you are sending signals, but there's no feedback," Abrash says. "What you can imagine is you combine EMG and gloves. And the point is not to literally emulate your hand. The point is that your hand is now a surface to provide feedback on."
But EMG is now closer to showing up in real tech, while haptic gloves are still stuck in the testing lab. "We'll probably have one input system that we'll just use all the time. I have no idea what it is, it could be everybody puts on their gloves in the morning, just like Victorians did -- you wear your gloves all day, and it has EMG as well," Abrash says. "But EMG is this very powerful channel for your mind to express its desires in a way that is completely frictionless. And to me in AR, it's just kind of hard to imagine how we're going to beat that in any near timeframe. It's technology that is on its way to product -- I can't tell you when, but it's not a 10-year thing. With haptic gloves, it is a research thing. And I can't tell you when it'll ship: It might be a 10-year thing. And it really addresses a different set of needs. It's not a way to express commands, it's a way to actually act in the world."
Meta sees EMG as something that will soon make its way into people's homes, in other words. But we won't see haptic gloves so soon.
Sean Keller leads AR and VR interaction and input research at Reality Labs Research.
The metaverse seems close, but it's not fully here yet
Abrash says his research group is focused on much farther-off goals than what Meta's consumer products team is working on, and we shouldn't expect to see haptic gloves around the corner. But he sees much of Meta's vision as a work in progress, and haptics and future inputs are a big part of that.
"We have been building, for over seven years, the pillars that any version of the metaverse will have to be built on," Abrash says of his further-off work in relation to Mark Zuckerberg's recent metaverse promises. "If we had good enough avatars right now, you and I might be doing this [chat] in VR, right? We're just building that underlying technology that the metaverse can be built on, and that's what we've always been doing. So the metaverse conversation in public came along, and my feeling was, "Great, I'm glad the world's catching up, it's a good thing we've been preparing for this."