Assembling the IT emergency kit

Much of the world is consumed watching the coverage of the enormous disaster that recently struck Japan. As if a massive earthquake and subsequent major tsunami didn't cause enough death and destruction, they unleashed a cascade of failures that led to serious nuclear power plant accidents that have yet to be contained, and that threaten lives and indeed the inhabitability of an entire area of Japan. It's simply horrific.

We humans think that we're in control of, well, everything. We have plans and lists and goals and policies and fallback positions. Then something like this comes along to reveal how tenuous our control really is. As the boxer Mike Tyson put it: "Everyone has a plan until they get punched in the mouth." This observation has previously been put more eloquently, but never more viscerally.

It's said that "every cloud has a silver lining." When a disaster kills thousands of people and imposes a gazillion dollars of economic damage, it's hard to see that lining. But if there is one, perhaps it's this: Other people's disasters lead us to re-evaluate--and just possibly, improve--our own preparedness. As we're watching in horror at how many protective systems failed or proved insufficient in Japan, maybe this would be a good time to think about our own protective systems, plans, and capabilities.

A great toolkit--This a great time for disaster-tolerant IT. All sorts of product advances in recent years make high availability, disaster tolerance, and business continuity--once hideously complex and expensive propositions available only to the richest corporations, and then only to their most important applications--more widely, easily, and affordably available. Virtualization, cloud computing, network computing, and distributed data stores are just a few highly leveraged enablers. Are you putting them to good use?

Defense in depth--"When it rains, it pours." All truly major failures are multiple disasters, layered one atop another. One system fails, then the backup fails, and the oops! becomes an OMG! When you're thinking about vulnerabilities and your backup, the next question should be, "OK, let's say that backup fails too. Then what?" And then prepare for that.

It's also not just about systems and apps, but also about the people and the environment we need to operate IT. September 11 sadly demonstrated that skilled people are an irreplaceable asset that, sadly, may be killed or injured. Even if they're safe, what if the staff can't get to your backup site? Or if it's not safe to operate there? What if electricity and/or network service aren't available? All these "secondary" considerations are truly primary.

Invest in safety and survivability--You can't possibly imagine everything that could go wrong. Nor can you possibly afford to insure against every possible failure. So you must make your best estimation of what might go wrong, what you can reasonably protect against, and what you can afford to spend to insure against those failures. Happily, approaches like system virtualization and network applications that reduce day-to-day costs are now directly leverageable when it comes to ensuring disaster tolerance. But it still takes resources and commitment to put together the "emergency kit."

Humans are famously lax until a disaster strikes--us or someone else--jarring us into motion. Developing quality backup systems is hard. But with IT running evermore of our businesses, our world, and our lives, also evermore essential. Japan's dark days give us reason to look to our own preparedness. Please take the opportunity to improve your safety and survivability should--God forbid!--disaster find your area.