The lessons of Sasser

CenterBeam CEO Kevin Francis says this security intrusion highlights fundamental weaknesses in the practice of software patch management.

Three weeks before the Sasser worm began to slither across the Internet, Microsoft published a patch to block the hole the worm uses to tunnel inside computers.

Nevertheless, the worm grounded at least 40 Delta Air Lines flights and delayed many more. The U.K. Coastguard was figuratively run aground and was completely offline for most of a day.

So, what happened? We had the tools to stop the worm dead in its tracks, but it still exacted a high toll in lost productivity, loss of real business and, in the case of the sailors at sea around the coast of England, created a real risk to life.

The root cause for this dysfunction can't be assigned to the lack of tools. We need to look deeper into the factors that contribute to the operational environment within information technology. This is where we might begin to understand why so many companies were left naked to the Sasser worm.

I think the most useful analogy comes in the form of a classic Greek myth. Sisyphus offends the ancient Greek gods, and he is condemned to forever roll a heavy rock up a hill, watching it roll down again and then rolling the rock back up the hill again.

When Microsoft released Office 2003, IT got to roll the rock up the hill, as it installed the new software. Two weeks later, the rock promptly rolled back down, when Microsoft issued the first "critical" patch for the program. The cycle subsequently got repeated, as each time a patch got applied, Microsoft would issue a whole new set of critical patches.

The root cause for this dysfunction can't be assigned to the lack of tools.
French existentialist Albert Camus used this myth as an emblem for his philosophy of courage, strength and even joy in the face of an apparently meaningless and futile life. Like the story of Sisyphus and his rock, there are far too many IT professionals devoting themselves to constantly patching systems. Their careers are entangled in the relentless chore of patching software instead of launching an enterprise resource planning or customer relationship management system that would create a competitive advantage and is already years behind schedule.

Of course, applying every patch is a chore more honored in the breach than in the observance. According to a recent poll conducted by CNET Networks', less than a quarter of those surveyed claimed that they applied every published patch. The vast majority said they needed to balance patch management with more important IT issues, and this is completely understandable--until something goes wrong.

The most simple-minded hacker doesn't even have to search for new ways to exploit a system. Instead, he watches for security patches, reads the documentation that explains why the patch is necessary and builds exploits for these holes, knowing that the odds are good that they'll be able to find--and compromise--unpatched systems.

The burden of patch management falls heaviest on the shoulders of IT staff at midsize companies. Large enterprises can reasonably scale resources to meet this chore.

Small companies simply don't have the resources to apply to patching, until the rock rolls back over them, and they're faced with a security breach or applications that just don't work. Midsize companies--and even some Fortune 500 companies--are trapped. They know the best practices. They know that they don't have the resources to implement a best practice. And, therefore, they know the deep, existential angst born from the futility of their situation.

Like the story of Sisyphus and his rock, there are far too many IT professionals devoting themselves to constantly patching systems.
IT managers take three broad approaches to successfully managing patches:

• Acquire software that needs less patching. It is very common to find companies avoiding the purchase of a ".0" release of software because of the inevitable patches, bug fixes and other loose ends that were left untied. Conservative software adoption is a good plan, but it merely delays the chore of patching.

• Become more efficient or get more resources to implement a rigorous patch management routine. But there's a cost in staff time and money. The IT team will need to constantly scour the market for new products, evaluate likely candidates, integrate the new tools into IT process management and train the staff. Besides assuming that there will be resources for the job, this approach takes as a matter of faith that people will make best use of what's available.

• Rather than shoulder the entire investment, a third approach is to share the cost of "best practice" patch management with outsourced IT management companies. Fixed costs such as the purchase of the software and staff can be amortized over a much larger population and substantially drive down the cost per seat. A company's IT staff can then remain focused on strategic IT issues and outsource the common, generic management chores.

An outsourced IT management company can be expected to have a completely different perspective on patch management than a corporate IT staff. For us, the failure to use the best possible tool is an opportunity cost; the failure to document the successful installation of every patch on every machine is a business risk, and the failure to staff this function with the best possible IT talent is just plain foolish. It's a matter of perspective.