Archiving - has this happened to you?
Has this ever happened to you in a small business?
Your manager has come rushing in, in a blind panic. Nothing new there then but you feel you need to do something about it, since he’s gone purple and it wouldn’t look too good on you if he keeled over in your cubicle.
Archiving is his topic du jour, has been on his radar (although he’s not sure why) for a couple of years and will be for a couple or three more. What your manager doesn’t describe is what he wants or means when he uses the word archive. Does he want to get rid of a load of mail because he’s just seen his cross charge from the SAN boys for mail storage? Does he mean he wants to take a copy of every mail coming into and going out from the Organisation just for the benefit of an as-yet intangible reason? Does he mean that he wants to track down mails that one user may (or indeed may not) be sending? Or does he mean he wants KVS, but isn’t entirely sure what a KVS is, what his requirements are, what the alternatives to this KVS are and what KVS could do for him – should he actually need the product.
What do you do?
Much has been written on archiving, but nothing seemed really appropriate for the SME market. Everything pretty much revolves around huge SANs, but you can archive on a smaller scale.
Taking this from basics, mail archiving is a multi headed beast. It can help your storage solution, it can help your compliance requirements and it can help your users in terms of searches and indexed. Not having it can cause you no end of aggravation with the regulators and having it could just as easily give you more. Worried? Good, now read on.
A great number of companies take a feed from their Exchange stores and push all mail coming into and going out from the stores into a Public Folder or to another mailbox store so that management, HR and other nefarious types can dip in as the fancy or permissions take them. That’s a fairly easy issue to overcome as it’s simply a tick-box on the properties of each store. It dumps an enormous amount of email into one or more folders, wherever they happen to be. It makes for an awfully slow searching operation and you certainly wouldn’t want to index it on a shared system, so you are into dedicated hardware territory. It allows you to effectively spy on people but it doesn’t help you one little bit when a message goes between two or more users on the same store. Doing this “Journaling” to give it the correct phrase is a good solution if you want to keep a weeks worth of unmodified work and then discard it.
Keeping it for any longer than a week is a different ballgame. So, you’ve got this huge amount of mail and it’s taking up a whole lot more space on your SAN (if you have one of course), your cross charged SAN bill just went through the roof or your Exchange Administrators are now in front of you with a begging bowl for more servers and storage, you have gained very little over and above a huge store that you can’t do a great deal with. Problem not solved really.
Most companies have their rogue employee who is sending all manner or inappropriate content, whether he originates that content or simply forwards it on. Put simply, it doesn’t matter how he got it, he’s sending it and your company has a duty of care over the other employees to protect them from all the rubbish that he (and certainly she’s) may be sending around. The solution to this one is easy. www.ivasoft.biz and get Selective Journaling. There isn’t really anything else to say about it so if you don’t journal mail as a matter of course and do employ a few porn fans or budding Jim Davidsons then it’s the best way of getting rid of them, assuming you have a written corporate policy on unacceptable use of email. Remember, you can’t get rid of an employee for having Gigabytes of porn in his inbox if there isn’t an administrative policy in place that says he can’t have it there.
What’s next? Well, there are a few low-end archiving solutions. GFI have one: http://www.gfi.com/mailarchiver/ and it does the job every bit as well as you might expect. You know Mail Essentials and Mail Security already, they keep your mail environment nice and clean, keeping out all the aforementioned porn and invitations to either provide various Africans and Russians with your bank details or dealing with all those tempting Viagra offers.
The problem of course with GFI’s solution (selected here merely as an example of many) is that it pumps all the mail into SQL. So now you have two enormous databases full of traffic, one in Exchange and another in SQL. All your investigators are crawling all over the SQL leaving Exchange to get on with being the best mail and calendaring solution on the market. How long are you going to keep this stuff in SQL and how are you going to give the users access to it? Well, you are going to keep it for as long as the cost to storage calculation works and then you’re going to send it to tape or perhaps even just get rid of it. Your users are going to access their old email through a web browser. Speaking personally I think that’s one hideously fragmented solution.
Do you need the Rolls Royce solution? The Roller solution need not necessarily break the bank. Yes it’s going to cost some money to put in, but the last three I’ve designed for customers actually had a negative cost on day one when it went in. Consider this:
You give all your users 100MB mail quota’s and have no archiving solution. What this means in reality is that because your Administrators have friends and people complain to their boss, most users have 200MB of mail in their mailboxes and about 400MB of PST’s floating around the network shares or local PCs. It’s a compliance nightmare, it’s a storage nightmare. The users corrupt their PSTs and expect you to get them back even though neither you nor they know where these PST files are. You implement Symantec Enterprise Vault (commonly still known as KVS) or an equally comprehensive solution and you get the following high level benefits:
• You introduce 20MB mail limits for all users, or select (say) 30 days as a maximum age for any message to be in an online Exchange store
• You eliminate PST files and use the Outlook GPOs to ban them from your world
• You store the archive solution on slower SATA disks rather than the Fibre Channel systems presently used
• Your backups for 10,000 users take a fraction of the time and rather than 4,000 users on your 4 way ProLiants you can get 6,000 or more on there.
• You implement the KVS Offline Vault so that your laptop users can see their archived mail that would otherwise be unavailable
• You have less mailbox servers
• You have less daily backups
• You have less expensive disks
• You have a lot less tapes because your Exchange contains so much less data and your archive only gets backed up on an infrequent basis.
• You have implemented a technology solution that also gives your FOI/Compliance folk a degree of comfort that they are able to control the information flow and respond to information requests, should the vultures lawyers descend on you.
So do you need a SAN for your archiving solution? No, you certainly do not, that’s the key point. A decent server with some local DAS will do you fine, this can be internal or this can be a standard external (read cheap) NAS array. It’s whatever suits your budget. The good thing here is that you know, pretty much, how much storage you are ever going to need on the Exchange server because anything over your 30 days is going to archive. You also know how much storage you want on your archive and because the archives are so well compressed you get more time or data on your archive that you do on your Exchange Server. You can also have a discreet tape cycle that only needs a weekly backup (because you’re only archiving weekly) You have three “known knowns” rather than “unknowns that we know about” and “unknowns that we don’t know”
Summary
If you are looking to upgrade your Exchange version or alternatively migrate to Exchange from some other 2nd tier solution such as Notes or GroupWise then you must include archiving at the very outset. This enables you to optimise your storage plans and put the right solution in from the outset. I do not do any Exchange designs for my company that do not encompass the Enterprise Vault product as a mandated component. I find that the entire storage management calculations can be given to a customer with a good degree of authority and the decision on storage can be made with cheaper disks and infrastructure, allowing the business to get on with making money.
I’m not employed by Symantec nor do I have any affiliation to them (over and above a KVS key fob) and there are other seriously good competitors. I use Enterprise Vault here as my frame of reference only.