The silent explosion of log management

Those mundane text messages produced by every conceivable technology device are growing like a proverbial weed.

What's the fastest-growing data source at large organizations? Video? Maybe at YouTube, but not at Citibank. The answer is log files. Yup, those mundane text messages produced by every conceivable technology device are growing like a proverbial weed.

Why the log file explosion? First off, every IT shop has more and more devices to manage each day, and more devices mean more logs. In the past, most IT managers paid limited attention to logs, but this too has changed. Driven by security monitoring, regulatory compliance reporting, and IT troubleshooting, large organizations now aggregate, analyze, store, and archive terabytes of log data every month. Indeed, the log management snowball is rolling down a very steep and very snowy hill.

It's certainly time that the IT industry recognizes that log management is no longer a tactical domain of IT administrators. Rather, think of log management as the foundation of a Network Information System (NIS). Analysis of log data (as well as other data sources like NetFlow and SNMP) is quickly becoming the difference between effective IT security/operations management and flying blind.

As log management gains strategic importance, look for the big guys to swoop in. EMC/RSA and IBM are already there, and HP is working with SenSage on its play. There are also several strong pure plays including ArcSight, LogLogic, and LogRhythm. All this said, where is everyone else? Log management seems like a natural for Cisco Systems, especially in concert with its whole Data Center 3.0 initiative. Symantec is also missing from the party thus far, as is CA, McAfee, Microsoft, and loads of others.

Logs seem trivial, and log management appears like a tactical task at the bottom of the IT stack. Maybe in the past this was true, but in today's world, information is power and logs are device-specific information. Aggregate these logs, add some intelligence, and provide tools for analysis, and you get a great temperature reading of what's going on across the IT system. Seems pretty important to me.

Featured Video
This content is rated TV-MA, and is for viewers 18 years or older. Are you of age?
Sorry, you are not old enough to view this content.

Top 5 Surprise Tech Hits

Sometimes a product may seem like a crazy idea, but it goes on to far exceed expectations. These are the most surprising tech successes.

by Iyaz Akhtar