Facebook weaves new network fabric
The social-networking giant is developing a new networking "fabric" to relieve bottlenecks within its data centers and speed deployment of new apps.
Facebook has a communication problem. The apps are talking to the network but with limited context and bottlenecks inside its data centers. Now, the social network is planning to improve the communication with a new networking fabric to serve its 1.1 billion users.
"Networking is built around ISP networks and protocols for interconnecting of ISP networks. It's not built around content and apps, so there are fundamental gaps in the requirements we have versus ISPs," said Najam Ahmad, head of networking at Facebook, speaking at the GigaOm Structure conference Wednesday. The gaps result in choke points that slow down apps and require weeks instead of minutes or seconds to deploy new features, he explained.
"The needs of our application are such that everything is virtually real-time and interconnected," said Jay Parikh, vice president of infrastructure engineering at Facebook. "Our traffic going across from machine-to-machine far exceeds the ratio of machine to user." He pegged the ratio at about 1,000 to 1.
As an example, one of Facebook's apps, involving page generation, requires 2.5 billion operations per second. "There's no way to compute between an app and the network, so the app puts a packet on a NIC [network interface card] and hopes it gets there," said Parikh. "We need technology that allows apps to have a better feel for what the network is doing."
Facebook's solution is to create more of a uniform, flat network architecture that would allow for easier network and app management and fewer choke points. Every cabinet or rack is at the same level and connections between any two racks are uniform, Ahmad said. This will allow Facebook to pass bits on a massive scale within its datacenters through a software-defined layer. "We manage hundreds of thousands of machines. For the network we have had a separate set of tools to manage them. Now we have same system to manage the network," he said.
The fabric is coupled with Facebook's Open Compute Project to create a more open-source, OS-agnostic hardware, including network switches, moving away from the tightly couple hardware and software solutions.