Understanding Unicorn: A dive into Facebook's Graph Search
Facebook engineers offer an inside look at the magical creature holding up the social network's new Graph Search platform.
MENLO PARK, Calif.--On the surface, Facebook's new Graph Search is touted as a new method to enable more connections between people and content worldwide.
It turns out what is making all of that possible is a Unicorn.
During a crash course Thursday at the company's headquarters, a team of Facebook engineers offered a more detailed glimpse at the magic behind the new, third "pillar" of the world's largest social network's strategy to enable new connections.
Soren Lassen, manager of the search infrastructure team at Facebook, described that "in computer science speak," the Social Graph is a graph data structure involving an "immense map" of people, places, and things and the connections between them.
"Graph Search was a particularly fascinating opportunity to do something new in the sense that we had a unique and different social set and structure," said Lassen.
With more than 1 billion active members, Facebook hosts approximately 240 billion photos with 350 million more added every day. The data only continues to expand exponentially with 2.7 billion likes and 2.5 billion items of content shared daily.
Lassen commented that "natural language is very expressive." Thus, one of the principle uses of Graph Search is to ask natural questions about the Social Graph through proper nouns.
Glossing over some of the technical jargon, Facebook software engineer Mike Curtiss continued that Graph Search was "designed from the ground up" to enable users to find nodes in the Social Graph connected to other nodes.
At the heart of Graph Search is this infrastructure dubbed "Unicorn," which is similar to a standard search engine in that it supports common search queries.
But Curtiss stipulated that Unicorn is unique because it supports multi-hop queries that are issued in a series of steps.
He highlighted that there are numbers connected to nodes, which are similar to keywords in a search engine. Curtiss explained they serve a similar role, but these nodes have a numerical structure to them connecting them to the semantics of the Social Graph.
Here's an example of how Graph Search works, according to Curtiss.
Say Curtiss is moving to New York City in a few weeks. He said it would be helpful for him to find which of his Facebook friends live there.
The Graph Search process starts with nodes for his friends and then traverses his friend connections, or "edges." Those individuals are connected to him through the "via-friend" edge and "lives-in" edge for New York City.
Curtiss expanded the query to "my friends who live in New York and like "Downton Abbey." From here, there was one individual that matched all three constraints: friend connection, lives in New York and likes "Downton Abbey."
"This is where things start to get really interesting," said Curtiss. "What I've shown so far is you can apply basic search techniques to the Social Graph. That's a cool idea in itself, but there's more that you can do."
Because there's a lot of inherent structure to a graph, Curtiss posited that "what would be really cool is if we help people find entirely new haystacks."
Curtiss explained that means Facebook wanted people to be able to search for both things that they are already connected to but also to items not directly connected to them.
"You'll notice that for this query we just did, we're always a single hop from the initial node that started the query," Curtiss commented. "There's often a lot of interesting queries that require more than one consecutive hop."
Curtiss extended the scenario further to suggest that if he's moving to New York, he might be looking for a job. Thus, one thing he could do is find out more about the employers of his friends.
"The way we would do that is within Unicorn. Start at the node for me, traverse the friend edge, and from those nodes, we can hop to the companies that my companies my friends work at through another edge," Curtiss explained.
This leads into the second step of Unicorn, which Curtiss referred to as the "Apply Operator," which takes a set of input results and applies another query to those results.
"It's kind of like the flux capacitor," Curtiss quipped, referencing the fictional technology from the film "Back To The Future" that supposedly made time travel possible. "The Apply Operator is what makes Graph Search possible," Curtiss added, being that the Apply Operator connects users to things to which they don't have a direct connection.
In the case of finding more employers, Curtiss explained how the Apply Operator opens the door from just finding the employers of friends to finding the employers of the friends of his friends who live and work in New York City.
However, Curtiss admitted that the problem with the Apply Operator is that the search results increase exponentially as you continue, which he argued makes sense when you think about simple math.
Curtiss explained that when you request edges of 100 nodes, that retrieves 10,000 output nodes, and then from there that requests the edges of those nodes to retrieve one million output nodes.
Simply put, Curtiss asserted you need to filter it down in some way. That's where social ranking comes in, and the direct friend connections (as well as more fine-tuned search terms) come back into play.
Yet Graph Search is still relatively new and not available to nearly all of Facebook's user base yet. In fact, it's only available in English to "hundreds of thousands" of Facebook users right now, and the team wouldn't reveal when access would be rolled out to everyone.
Citing one growing pain on the agenda, Lassen noted that in order to index all of this data, the engineering team needs to do "some fundamental scalability work."
While acknowledging that Graph Search still has a long way to go, Curtiss said that the team has some "clever ideas" about how to make it better, adding that CEO Mark Zuckerberg has expressed that this is a project he really cares about.
Lassen concluded, "This journey is one percent done, which means we realized there's an enormous opportunity to do more in this space."
This story originally appeared at ZDNet's Between the Lines under the headline "Understanding Unicorn: A deep dive into Facebook's Graph Search."