Your future face-scanning phone could check for live skin
Sound creepy? It may actually make your Face ID-type sensor more secure and accurate. The company behind this expects to see the tech in phones by 2021.
- Nearly 20 years writing about tech, and over a decade reviewing wearable tech, VR, and AR products and apps
An infrared dot projector and camera sensor combine to give Trinamix' tech what it needs to sense different materials (like living skin).
Fooling a face-scanning phone camera may be a lot more challenging in 2021. Live-skin-sensing technology is coming to phones , promising a way to 3D scan your face -- and also check to see if your face is made of actual living flesh.
Trinamix, a subsidiary of German chemical company BASF SE, promises it could work with a variety of affordable components, too, and even end up in affordable global phones as soon as next year. Trinamix has already partnered with Qualcomm to work its algorithms into future 3D face-scanning phones. The tech uses existing infrared dot-projector and camera sensors, along with new algorithms.
Detecting living skin sounds completely creepy, but it's a practical idea if you think of it as an extra form of verification before completing a facial ID scan. Face ID-type sensors found in phones like the iPhone 11 and Google Pixel 4 use a 3D map of infrared dots for security, and in some cases a 2D photo as well. Trinamix will also check backscatter on the reflected infrared and be able to identify whether the face is living skin. And yes, dead skin will provide a different backscatter, according to Trinamix founder and CEO Ingmar Bruder.
In a midtown Manhattan hotel suite, Bruder guided me through a few demos of the tech and how it works. A couple of green blocks and a rubber human face mask lay in front of an array of cameras. One camera was cabled up to a laptop, while the other sat atop an LG phone.
Testing my face using Trinamix tech. Live skin, sensed.
The phone demo showed what would happen if the rubbery face mask were held up to the camera -- the prototype app's "live skin" check box didn't activate. My face, or Bruder's face, passed the test, however. I found I had to get my face in a very particular angle for the early demo to work, and it remains to be seen how the demo can handle serious attempts at spoofing (the rubber face mask we had was nowhere near like our faces, but Bruder said a Mobile World Congress demo would have shown a more advanced comparison with a realistic face mask).
The laptop demo showed how Trinamix's camera algorithms could recognize different materials. For example, two green blocks in front of me looked similar, but one was made of plastic and the other of wood. A display on the laptop with a pixilated readout of the camera's IR dot backscatter showed one block as differently colored than the other. That's the program's way of labeling what's perceived as different materials. My hand, which was also in the camera's view, showed my fist as pink, but a bluish area where my shirt sleeve began. Trinamix could adjust the algorithm to look for different materials and identify wood, metal, living skin, a car cabin, a seat belt and more.
Two different blocks that are the same color: the camera's scan on the laptop color-codes what it perceives as different materials.
The way Trinamix recognizes materials through its camera array has to do with identifying the backscatter of infrared or lasers that are bounced off a 3D scan. Imagine Apple's Face ID, which sends an infrared array of dots out, but now think of a program that can study the light from those dots afterwards and make additional sensor readings.
Trinamix's tech works by individually measuring each infrared or laser dot's distance individually, which is a different technique from existing infrared dot-array facial scanning that looks for a distortion in the overall pattern of dots. But, more importantly, this tech could be applied to any phone, laptop or other 3D face-scanning camera system, using cameras from various suppliers. The goal is to make the technology affordable enough to include in all levels of devices.
I tried using a rubber mask, which didn't trigger the "live skin" detector.
Trinamix's early camera prototypes use a small Raspberry Pi computer that works with the LG phone's USB-C port, but the actual camera array should use only the phone's internal processor, and rely on similar infrared dot projectors and camera sensors like existing face-scanning phones, though aligned in a way that will work with Trinamix' algorithms. Currently, the technology will work with Qualcomm-powered Android phones. The company isn't working with Apple at the moment, said Bruder.
The applications go way beyond extra levels of face-scanning security. Trinamix sees this tech being helpful in cars to detect whether a living driver is sitting down or whether a seat belt is fastened. It could be used for factories or warehouses where robots need to look for and pick up particular objects by material, Bruder said. Maybe, by identifying materials, it could help computer vision programs train faster.
It's an unsettling idea to imagine my phone identifying my living skin, but as a safety and security feature that could possibly make my phone handle facial recognition-based security better, it makes sense. As to how well it will work on a phone, and how it will function with a manufacturer's particular software, it's hard to tell yet.