big data

Twitter's latest buy: Big data startup Lucky Sort

Twitter's shopping spree shows no signs of letting up. Today, the social-networking giant said that it has acquired big data startup Lucky Sort.

Lucky Sort CEO Noah Pepper said in a blog post that his company's "goal was to make huge document sets easier to analyze, summarize, and visualize by building elegant and user-friendly tools for text analysis."

Neither Twitter nor Lucky Sort said precisely how the startup will be integrated into Twitter's larger plans, but Pepper said in his post that several members of his team would be moving to San Francisco to become … Read more

RSA sees 'big data' as key to corporate security (podcast)

Big data is one the the big themes at this week's annual RSA security conference in San Francisco.

That's because analyzing a company's stores of data is another step in improving information security, RSA Vice President Brian Fitzgerald said.

"Classic security defenses are no longer that effective in a world where data centers no longer have a fixed perimeter. They're connected to suppliers and to customers. Information is flowing between partners on a massive scale," he said in an audio podcast (scroll down to listen).

Analyzing a company's data allows you to "… Read more

Big data gets its own book: 'The Human Face of Big Data'

Big data, one of tech's biggest buzz phrases of the moment, is about to get its own book. Fittingly, a really big, 7.5-pound book.

Called "The Human Face of Big Data," this is the latest project by longtime photojournalist Rick Smolan, the one-time National Geographic photographer who's best known for creating the "Day in the Life" series of books.

Smolan's approach for this work was the same as the one he's used for his main prior projects: He dispatched an small army -- in this case 100 photographers and 22 researchers … Read more

The post-election tech tally: Winners and losers

Elections are all about winners and losers, who is up and who is down. Here's a CNET look at the winners and losers in the 2012 election in which President Obama bested former Massachusetts Gov. Mitt Romney, from a tech perspective.

WINNERS

Nate Silver: The FiveThirtyEight forecaster's algorithms correctly calculated that on election day President Obama had a nearly 92-percent chance of winning, and accurately projected the voting outcome in 49 states (Florida has not yet been called).

Read: Obama's win a big vindication for Nate Silver, king of the quants

Big data: Many, especially … Read more

Beyond hashtags: Instagram's next challenge is tackling big data

SAN FRANCISCO -- Instagram CEO Kevin Systrom said his company's big hurdle is knowing how to mine the constant stream of photos being uploaded each day to the popular network.

"Instagram isn't necessarily a photo company, or a communications company as I like to say, we're also going to be a big data company," he said today at the GigaOm Road Map conference in San Francisco.

Systrom said the company, recently purchased by Facebook, needs to find better ways to organize the billions of photos shared on Instagram. That means looking beyond hashtags for looking … Read more

How tech is changing the road to the White House (video)

We've come a long way since the iconic 1964 "Daisy Girl" political ad. These days, candidates are turning to complex data sets to help pinpoint potential supporters.

So exactly what kind of information are political parties getting about you? According to a New York Times article, callers will know if you have homes in foreclosure, what kind of beer you drink, or the type of vacations you enjoy.

But technology works both ways.

Keep the facts straight with apps like Super PAC and Ad Hawk. These free apps "listen" to political advertisements on television and … Read more

Latency matters in a hybrid cloud

"There's that pesky speed of light." That cautionary remark was offered by Lee Ziliak of Verizon Data Services, speaking on a panel at the 451 Group's Hosting and Cloud Transformation Summit last week. The context was that hybrid cloud environments may logically appear as something homogeneous, but application architectures need to take the underlying physical reality into account.

Latency, the time it takes to move data from one location to another, often gets overlooked in performance discussions. There's long been a general bias toward emphasizing the amount of data rather than the time it takes … Read more

Data isn't always the answer

"Big Data" promises to turn terabytes, petabytes, and exabytes (with, presumably, zettabytes and yottabytes to come) of what's often ambient digital detritus into useful results. That promise often seems to come with an implicit assumption; with enough data and the tools to crunch it, useful insights will follow. Insights that can be used to make businesses more efficient, tailor everything from medicine to advertising for individuals, and employ instrumentation and automation on larger and more complex physical systems than ever before.

For example, we're in the early days of what sometimes goes by the name of … Read more

Open source powers big data index

Interest in big data continues to grow in terms of both downloads of connectors to software packages and in software infrastructure to power big data, primarily in the form of NoSQL databases and Hadoop-related extensions, according to a report.

The report, released today to coincide with the Hadoop Summit in Santa Clara, Calif., comes via open-source business intelligence provider Jaspersoft. The second-quarter report measures demand for popular data sources for storing, analyzing, and visualizing big data and uses stats from the JasperForge community site.

Key findings:

Big-data downloads are on pace to grow 92 percent in 2012 compared with 2011. … Read more

VMware works to make Hadoop 'virtualization-aware'

VMware today announced a new open-source project called Serengeti, which enables enterprises to quickly deploy, manage, and scale Apache Hadoop in virtual and cloud environments.

VMware says it is working with the Apache Hadoop community to contribute extensions that will make Hadoop Distributed File System (HDFS) and Hadoop MapReduce projects "virtualization-aware" to support elastic scaling and further improve Hadoop performance in virtual environments.

In case you've been living outside the big data vacuum, open source Hadoop has emerged as the de facto standard for big data processing and is packaged up in a few different distributions by … Read more