Backing up e-mail

E-mail, for many of us, is very important and accumulates forever, making it a large mess when it comes to backing it up.

The importance of my e-mail snuck up on me. Once upon a time, I opened my old reliable e-mail program and was confronted with an error message. The net effect of the problem was that the last four days of incoming mail had disappeared from my in-box. This was, for me, a very big deal. In large part, my in-box is my "to do" list. As a consultant, my incoming e-mail is too important to ever allow a repeat of this problem.

Suffice it say, this made me think about backing up my e-mail perhaps more than most people.

The need for reliable and redundant e-mail backups dictates the use of a client side e-mail program such as Outlook Express, Thunderbird or Eudora. Web based e-mail systems such as Gmail, Yahoo mail and Hotmail, have their advantages but backup is not one of them.

To begin with, I have an external hard disk attached to my computer and every morning I copy all of my e-mail from the internal hard disk to the external one. This is a destructive backup. That is, every morning the backup is totally re-created on the external hard disk. The advantage of this is that I never have to worry about running out of space on the external hard disk. The disadvantage is that I can't use it to recover e-mail from three days ago. Everything is a trade-off when it comes to backups.

Also, this backup doesn't manipulate the original files in any way; they aren't combined, compressed or re-formatted. Thus, I can easily copy e-mail from the external hard disk back to my computer and use it immediately. And simple means there is less that can go wrong. The downside is that the backup is the same size as the original, but external hard disks have a huge capacity and transferring files over a USB2 connection is more than fast enough for this purpose.

One of my prime rules for backups is to never to copy a file while it's in use. That is, I never copy e-mail when my e-mail program is running and never copy Word documents when Word is running. The morning backup of my e-mail is scheduled by the Windows scheduler and since it runs first thing after Windows starts up my e-mail program is not running.

This however, is just a starting point as it still allows for the loss of an entire day's worth of e-mail. To cut my potential loss in half, I also backup my e-mail midday. This backup is also scheduled using the Windows scheduler, but it's very different from the morning backup. Rather than backing up all my e-mail, here I only copy the most important folders (the in-box and a few others). Also, the backup is sent via FTP to an online file storage company.

This limits my worst case scenario to the loss of a half day's worth of e-mail. It also means that no matter what happens to my computer and the external hard disk, I always have the most important e-mail stored a thousand miles away. And since my e-mail is sensitive, online storage space is limited and uploads are slow, I compress, encrypt and password protect the e-mail before it leaves my computer and travels over the Internet to the file storage company.

The midday backup is different in other ways too. For one, all the e-mail is combined into a single file. In addition, I keep multiple copies of the midday backup. The backup program tags the daily file with the current day of the week. Thus every backup made on a Monday will result in the same file name. When the backup is sent offsite, the backup program is instructed to delete older versions of files with the same names. I end up with seven off-site copies of my most important folders and, again, don't have to worry about running out of space.

Finally, once a month I compress and encrypt all my e-mail and send it off-site to another file storage company.

No one approach is right for everyone. For example, I have chosen to limit my worst-case loss to a half day of e-mail, which may not work for you. And my approach requires constantly filing e-mail in folders, something not everyone wants to do.

After living with the above scheme for a while, I modified it a bit to prevent the most important folders from growing in size forever.

I manually archive the in-box, sent folder and a few other important folders by moving old messages to new folders tagged with the year. For example, all the messages in my in-box from 2005 are stored in a folder called inbox2005. Likewise there are folders called inbox2004, inbox2006 and inbox2007. A couple months ago I moved messages in my in-box from January through March of this year into the inbox2007 folder. Later this year, I'll again move old messages from this year into it.

With this approach, I can eventually delete the inbox2004 and inbox 2005 folders from my computer. They remain on the external hard disk and are also stored off-site if need be. Without some type of archiving scheme, e-mail will grow forever. I find that manipulating a few folders this way a couple times a year is well worth the effort.

Of course, you can't use this approach, or anything remotely similar, unless your e-mail program stores each folder as a separate file (or two). But who would use an e-mail program that stored all your mail in a single file? :-)