Facebook's AI tech now helps anyone build computers that can see
The technology, available for free, lets computers isolate subjects in a photo. That could be a help to blind people, self-driving cars and beyond.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
When you look at a scene, your brain makes sense of all the elements -- trees, handwriting, your biking buddy Josh -- with an ease that puts even the most advanced computers to shame.
Now Facebook is sharing artificial intelligence technology that helps digital vision come a step closer.
The company said Thursday it has developed technology that can figure out the boundaries of objects in a photo, a task called segmentation, and released it as open-source software that anyone can use or modify for their own ends.
It's not the kind of thing you'll install on your PC or find on an app store. But by sharing this with any programmer, Facebook could help bring computer vision to all kinds of projects: drones that avoid collisions, camera apps that write their own photo captions, instruments that spot problems in medical scans, and self-driving cars that know the difference between a fire hydrant and a child.
A decade ago, this kind of technology would have been a closely guarded company secret. No more.
"It's our hope that others will be able to work with us to improve our tools and technologies," said Piotr Dollar, a research scientist with Facebook Artificial Intelligence Research. "Open collaboration helps foster innovation for future designs, putting us all one step closer to building complex AI systems that bring this kind of innovation to our users."
Facebook illustrates aspects of computer vision.
It's not the first such sharing move. Facebook and Google each released their core neural network software projects -- Torch and TensorFlow, respectively -- that offer a general-purpose foundation for machine learning and artificial intelligence. Google has open-sourced its SyntaxNet system for figuring out sentence structure, too, and Facebook did the same with its FastText and DeepText projects for classifying and understanding text.
AI is one of the hottest things going in tech these days, and the driving ambition is a huge one: building computer systems that can learn and adapt on their own, much as humans do. Forrester Research predicts that within a few years, companies will spend billions of dollars annually on artificial intelligence.
The goals aren't all on a grand scale. Facebook CEO Mark Zuckerberg wants to teach himself to code an AI assistant -- think Jarvis in the "Iron Man" movies -- that can help him at home and in the office.
With its newest move, Facebook released and documented three linked projects in research papers. First is DeepMask, a segmentation tool that roughly outlines objects in a photo; SharpMask, which refines the outline; and MultiPathNet, which tries to identify objects based on those outlines.
Facebook has demonstrated its computer vision technology as a way to describe images to blind people. But the company wants to go beyond that.
"Our goal is to enable even more immersive experiences that allow users to 'see' a photo by swiping their finger across the image and have the system describe content they're touching," Dollar said.
Right now researchers typically "train" neural networks by feeding them immense sets of data, a task that requires lots of powerful computers. But once the training is done, the AI knowledge can be distilled into a model that'll work on more modest hardware.
"Getting these models to run in real time on mobile devices continues to be a really interesting and active area of research for us," Dollar said.