Fooling a face-scanning phone camera may be a lot more challenging in 2021. Live-skin-sensing technology is coming to phones, promising a way to 3D scan your face -- and also check to see if your face is made of actual living flesh.
Trinamix, a subsidiary of German chemical company BASF SE, promises it could work with a variety of affordable components, too, and even end up in affordable global phones as soon as next year. Trinamix has already partnered with Qualcomm to work its algorithms into future 3D face-scanning phones. The tech uses existing infrared dot-projector and camera sensors, along with new algorithms.
Detecting living skin sounds completely creepy, but it's a practical idea if you think of it as an extra form of verification before completing a facial ID scan. Face ID-type sensors found in phones like the backscatter on the reflected infrared and be able to identify whether the face is living skin. And yes, dead skin will provide a different backscatter, according to Trinamix founder and CEO Ingmar Bruder.and use a 3D map of infrared dots for security, and in some cases a 2D photo as well. Trinamix will also check
In a midtown Manhattan hotel suite, Bruder guided me through a few demos of the tech and how it works. A couple of green blocks and a rubber human face mask lay in front of an array of cameras. One camera was cabled up to a laptop, while the other sat atop an LG phone.
The phone demo showed what would happen if the rubbery face mask were held up to the camera -- the prototype app's "live skin" check box didn't activate. My face, or Bruder's face, passed the test, however. I found I had to get my face in a very particular angle for the early demo to work, and it remains to be seen how the demo can handle serious attempts at spoofing (the rubber face mask we had was nowhere near like our faces, but Bruder said aa more advanced comparison with a realistic face mask).
The laptop demo showed how Trinamix's camera algorithms could recognize different materials. For example, two green blocks in front of me looked similar, but one was made of plastic and the other of wood. A display on the laptop with a pixilated readout of the camera's IR dot backscatter showed one block as differently colored than the other. That's the program's way of labeling what's perceived as different materials. My hand, which was also in the camera's view, showed my fist as pink, but a bluish area where my shirt sleeve began. Trinamix could adjust the algorithm to look for different materials and identify wood, metal, living skin, a car cabin, a seat belt and more.
The way Trinamix recognizes materials through its camera array has to do with identifying the backscatter of infrared or lasers that are bounced off a 3D scan. Imagine, which sends an infrared array of dots out, but now think of a program that can study the light from those dots afterwards and make additional sensor readings.
Trinamix's tech works by individually measuring each infrared or laser dot's distance individually, which is a different technique from existing infrared dot-array facial scanning that looks for a distortion in the overall pattern of dots. But, more importantly, this tech could be applied to any phone, laptop or other 3D face-scanning camera system, using cameras from various suppliers. The goal is to make the technology affordable enough to include in all levels of devices.
Trinamix's early camera prototypes usethat works with the LG phone's USB-C port, but the actual camera array should use only the phone's internal processor, and rely on similar infrared dot projectors and camera sensors like existing face-scanning phones, though aligned in a way that will work with Trinamix' algorithms. Currently, the technology will work with Qualcomm-powered Android phones. The company isn't working with Apple at the moment, said Bruder.
The applications go way beyond extra levels of face-scanning security. Trinamix sees this tech being helpful in cars to detect whether a living driver is sitting down or whether a seat belt is fastened. It could be used for factories or warehouses where robots need to look for and pick up particular objects by material, Bruder said. Maybe, by identifying materials, it could help computer vision programs train faster.
It's an unsettling idea to imagine my phone identifying my living skin, but as a safety and security feature that could possibly make my phone handle facial recognition-based security better, it makes sense. As to how well it will work on a phone, and how it will function with a manufacturer's particular software, it's hard to tell yet.