Facebook processes more than 500 TB of data daily

The site manages millions of photos and processes billions of likes each day. That's a whole lot of sharing.

Jay Parikh runs Facebook infrastructure Facebook

Since Facebook uses this data to build its user experience, it wants teams from across the company -- whether they sell ads or build functions -- to be able to access any of the data as needed. Parikh said this keeps the creation and improvement of Facebook features as fast as possible.

A function like friend recommendations, for example, needs constant data updates, so that when you add a new friend, you see those connections immediately, Parikh said.

These nearly real-time efforts apply to most functions throughout the site because people won't use the site if the personalized experience is poor, or slow, he said.

"We can't afford for your photo be be uploaded and stored next week," Parikh said.

Instead of partitioning the data -- essentially dividing it up and storing it based on criteria -- like most companies do to make data more manageable, Facebook keeps it in one place for easy access.

That means an engineer who wants to identify stats or trends in a function, like how quickly people respond to messages, can easily get the data, write a code, and get results.

When pressed by reporters, Parikh said Facebook has a zero-tolerance policy when it comes to any abuse from this broad access. Additionally, all access is logged and monitored heavily, he said.

If you want to see Parikh's short presentation and a flow chart of its data system, see below.

Updated at 3:03 p.m. PT: with more info and a slideshow.


Join the discussion

Conversation powered by Livefyre

Show Comments Hide Comments
Latest Galleries from CNET
The breathtaking art and science of light
SkyBell's Video Doorbell chimes in (pictures)
Here's looking at you, Cloud LED (pictures)
Take a peek at Microsoft Word, PowerPoint and Excel for Android tablets (pictures)
Twitter's iOS app gets cool new features (pictures)
An iPhone 6 case designed for Moment lenses (pictures)