Is an antivirus gap looming?

My background is in information security (exploit code, software vulnerabilities, intrusion detection and network malware), as is the background of some of my friends. We have spent a lot of time over the past few years working on things like IDS signatures and the consequences of getting them wrong.

As such, many of my infosecurity friends look at the state of antivirus signatures and wonder why variants can't easily be detected. Especially in the first few hours of a massive malware outbreak, the failure to detect leaves many of us frustrated. When my friends in security operations are facing an outbreak, time is of the essence, and their patience for signature updates has worn thin.

These two fields have rarely had to meet in the past. Infosecurity researchers rarely applied rigorous academic studies to malware, seeing malcode as an uninteresting topic. Antivirus researchers rarely concerned themselves with vulnerabilities and exploits. Despite this history, we're beginning to see a convergence of the two, one that will probably be totally complete in the next five years.

Some of this is due to how the online world has changed this decade, and how a lot of malware has taken advantage of security flaws that can only be discovered through infosecurity research.

But some of this convergence is about filling the gap left by the antivirus companies. This gap between problems and solutions is twofold. First, it's about coverage of malcode samples and threats that they face. AV simply doesn't detect enough of the things out there on the front lines. The second gap comes from the response time. Fed up with the delays, the infosecurity community has taken action to fill the void.

While no one would openly suggest running a computer without some form of antivirus tool, most agree the protection it offers is increasingly disappointing. When I analyze malware I typically find variants of well-worn families with names like "Banker," "SDBot" and "Peacomm," yet most fresh samples aren't detected by most antivirus vendors.

Anyone tasked with helping to protect a user base from common Internet threats has seen this and is increasingly frustrated with antivirus methods. So why do we still insist on telling people to scan their computers with updated antivirus tools when we know the odds that all of the malware will be identified and removed is marginal? It's because we don't have better options, yet.

The antivirus world first developed in an era of poorly connected users, when viruses spread over floppies and file downloads. The world had fewer virus authors, and far fewer virus users, or people who modify others' virus software for their own use. In short, time was a luxury that everyone had in abundance.

This isn't the case any longer, and it hasn't been for several years. The time that the antivirus industry had to turn around signatures and disperse signatures to those in need has dwindled from a day to a couple of hours. Couple that with the large number of minor variants that appear for almost every family means that traditional signature-based antivirus is under duress, it simply fails to meet the needs of network security operators in the current threat landscape.

The frustration felt by many in the network and desktop security operations world is palpable. There have never before been so many users of malware analysis tools like multi-antivirus scanners like VirusTotal and Jotti, sandboxes like Norman, Sunbelt and now Anubis, and so many outsiders testing antivirus software. The pressure on the antivirus firms is visibly increasing, and they're going to have to do something about it.

The failure of antivirus companies to adapt to the dramatic malware appearance rates in 2007 tells us there's time for a change and there's room for a new class of tools. "AV is dead" is the battle cry of a new industry analyst report. Antivirus companies may not be going the way of the dodo, but to many customers, the concept of antivirus as the last line of defense has been thrown out the window. It's time for a better approach, one that can keep up and really defend networks.