Facebook AI paints better word pictures for the blind
An artificial intelligence upgrade means the social network's computers better understand photos. That helps image search, too.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
Blind users of Facebook just got an upgrade in the social network's ability to explain photos to them -- and the same technology helps everyone else, too.
In brief, Facebook's artificial intelligence technology now uses verbs, not just nouns, in the photo captions that computers can read to blind people.
"Until recently, these captions described only the objects in the photo. Today we're announcing that we've added a set of 12 actions, so image descriptions will now include things like 'people walking,' 'people dancing,' 'people riding horses,' 'people playing instruments,' and more," said Facebook's applied machine learning leader Joaquin Candela, in a blog post Thursday.
That technology also helps others among the 1.23 billion who use Facebook daily. That's because, as with Google Photos , you can now search for photos based on Facebook AI's assessment of their content, not just based on how humans happened to describe the photos with text when they posted them.
AI has long held promise to improve computing -- indeed, the term artificial intelligence was coined 60-odd years ago. But only in recent years has it begun to deliver. And boy, has it. Expect major changes this year well beyond tech companies as neural networks reshape everything from the drudgery of setting up meetings to the creativity of logo design.
Candela detailed the work at the Machine Learning @Scale conference in New York.
Facebook's AI uses neural network technology, too. It begins with the mammoth computational chore of training. For the object recognition used in Facebook's image search, the AI system was trained with tens of millions of photos, each annotated by a human in advance so the system could learn which pixel patterns correspond to which subjects.
The caption-reading technology was similarly trained, though with a smaller set of 130,000 public photos shared on Facebook. Humans annotated each photo so the AI system could learn what actions are shown.
Solving for XX: The industry seeks to overcome outdated ideas about "women in tech."
Special Reports: All of CNET's most in-depth features in one easy spot.