The joys of real-time data analysis for online retailers

Re-reading a piece I wrote a few weeks back about the uptick in online sales during Black Friday, I started to wonder if real-time customer intelligence is what is driving online retail growth.

There are undoubtedly a number of aspects to the growth in online sales. But after spending some time with a few of the major online retailers last week--including one who might not be considered a "retailer" in the traditional sense, I realized that the online world has a huge competitive advantage in its predilection toward data analysis with actionable near real-time results.

Amazon's suggested items and Apple's accessory push over the holidays are basically "we know this, so we suggest that" approaches toward customer loyalty that have been very successful. But taking advantage of what we know via programmatic interactions between human and browser or mobile phone is being greatly extended into gaming services such as Zynga, which can very specifically target a user with an upsell or offer new item based on the analysis they perform in the course of gameplay.

The biggest challenges are knowing what to look for and being able to use the findings in ways that have immediate bearing on revenue generation. The challenge is that to take advantage of the data you need to have both technical and data science skills--which are hard to come by these days.

Accordingly, there will be more and more software-as-a-service (SaaS) offerings that provide data intelligence and analysis. But the issue of what you should be looking for remains a challenge.

From a marketing perspective for example, do you care about lead conversion, product mentions on Twitter, or shopping cart abandon rate? If you are watching your infrastructure, do you care about Amazon Web Services (AWS) images stalling, your service-provider expenses, or network latency? There is an enormous list of data sources to track and measure, but what matters to you may be very different from what matters to the rest of your team.

In a recent Webinar, Cloudera CEO Mike Olson suggested that big data needs to become more about what problems Hadoop and other tools can solve for me and how can I deploy and use them easily. Further to this point is that it's not about how much data you have, but what kind of data you have and what you do with it.

Online retailers have excelled in finding data and knowing what to do with it, but brick and mortar stores still have a wealth of other data, including customer sentiment, patterns related to sales, location-mapping in store, and other psychographic data. But the real trick is being able to use the data in near-real time, as opposed to a historical view--which in today's world may just be a few hours or days old.

This opens the door to what I have often referred to as the analysis of everything--wherein we look at every aspect of our operations to increase efficiency and eek out more profit.

To that end, as you sit around over the holiday break, there are two important tasks you can undertake: analyze your own company and/or start a new company that takes advantage of this data. I'll be waiting to hear your results.