Optimizing Outlook 2007 Cache Mode Performance for a Very Large Mailbox

[The_Exchange_Team]

My mailbox size is 7.4GB. I send/receive around 200 emails a day; so that 7.4GB of email goes back several years. I don't use too many rules so most of my email ends up in my Inbox (30k items). If you have read Exchange mailbox sizing guidance, you will notice that my mailbox size is higher than the recommended max OST size (2GB), and I store more items in my Inbox than the recommended (5k). Since I am one of the contributors to this guidance, I felt I should come clean (by writing this blog); and explain how I use my email that meets the following criteria:

• Large Mailbox (several years worth of email stored in the Exchange Store)
• No PST usage
• Good laptop cached mode user experience
• Good workstation/home computer cached mode user experience
• The ability to search my entire mailbox
• My email processing does not over burden the server

Before I get started, I should probably tell you what hardware/software I am using

Software	Office Workstation	Home Computer	Laptop
Windows Vista SP1 RC1 (x86)	2 core 2.66ghz	2 core 3.2ghz	2 core 2Ghz
Office 2007 SP1	4GB Memory	2GB Memory	2GB Memory
	10K SCSI disk	7.2k SATA disk	5.4K SATA disk

*The software is the same across all machines. **The disks sited are those which host my Outlook 2007 OST (I am the Exchange Storage Program Manager; I have lots of disks in all of my computers).

Not surprisingly, I have a different user experience processing my mailbox depending upon which machine I am on. Also not surprisingly, my experience has changed as my mailbox size has grown. As my mailbox began to exceed 2GB, I noticed that my laptop experience began to degrade. It took longer to sync down new messages, longer to do sorts etc. As my mailbox exceeded 4GB, I noticed the same thing on my home computer. I now experience the same slowness on my office workstation with my 7.4GB mailbox (don't ask what my laptop experience is like). This is expected since each machine has a different performing hard disk (which my Outlook 2007 OST is stored on) and the fact that Outlook storage performance will slowly degrade as the mailbox size increases. Outlook development folks have the same max mailbox size as me (~10GB), so they are also very much aware of the performance issues associated with a large OST; and are working toward addressing the issue in future versions of Outlook. They made great progress in improving OST scalability with Outlook 2007 Sp1, but I hear there is still work to do. My background is in software performance (especially storage performance); so I couldn't live with the status quo. I was stuck, I needed my large mailbox; but I also needed a great/fast user experience so I could effectively process my mailbox. All of my machines had sufficient memory (2GB or more) so I couldn't solve the problem by adding memory (to provide the Windows System Cache with more memory to buffer the OST IO's). I basically had the following options: 1. Increase the IO performance for the OST. This would involve doing one of the following:

a. Installing the fastest hard disk available (15k 2.5" SAS). This would require a new disk controller. b. Building a RAID10 array of several disks. Since I am a storage guy, the RAID10 approach (combined with a nice battery backed write caching array controller) was rather appealing. c. Installing a flash disk. Flash is still a bit too expensive and the write performance does not match the read performance (not even close) on today's devices. I wanted a good user experience both when Outlook 2007 was syncing down lots of new messages, as well as when the OST was up to date.

2. Run Outlook 2007 in Online Mode. This moves the IO burden from the client to the server and doesn't address the disconnected laptop scenario. 3. Limit the amount of email synced to my OST. Outlook 2007 has the flexibility to limit what folders get synced to your OST. You can reduce the size of your OST by configuring sync policies on a per folder basis. By configuring this, I could have my 7.4GB mailbox but only sync down a subset (say 2GB) to my OST. Option 1 was what I was originally targeting but it didn't make much sense to solve the IO problem only on my workstation and home computer (2.5" SAS disk or RAID array); but not solving the problem on my laptop. A flash disk would work for all my computers, but I wasn't willing to invest that much cash to solve this problem; nor was I happy with the random write performance of these devices. Option 2 was a non starter since it breaks for my disconnected scenarios (Wi-Fi is not quite ubiquitous). I could use Online Mode for my workstation and home computer; but I would be adding additional burden to the server in the process; and still wouldn't have a laptop solution. This left me with Option 3; limit the amount of data I sync to my OST. This is what I did. Step 1: Designate a small set of "archive folders" and move old items in to them (or us MRM to automate this) We utilize MRM (Messaging Records Management) policies in our Exchange 2007 deployment. The policy is configured like this:

• Any email older than 1 year gets moved to "Managed Folders\Will Expire" folder. I have 90 days to move any email moved to this folder to another "Managed Folder" with a longer expiry policy before it gets deleted.
• "Managed Folders\Research and Development" is a Managed Folder that has no expiry policy. I move messages in to this folder that are related to Research and Development that I want to keep longer than a year. The majority of my email relates to R&D so the majority of my email makes it in to this folder eventually. To help organize the folder, I have created subfolders based on year beneath this folder:

Managed Folders Research and Development (No Expiry) 2004 2005 2006 2007

Personally, I manually manage moving old R&D messages in to the correct R&D managed folder (messages from the year 2005 go in to the 2005 folder etc...) MRM could also be configured to do this for me (and for everyone else in the Organization for that matter), but would not provide the flexibility of picking and choosing which messages get moved to the Managed Folders. If you wanted to move all messages of a given age for all users to a given Managed Folder, an MRM policy could easily be configured to do this. This is ideal for organizations who have unlimited expiry policies and want to provide the ability to easily mass move old items out of the primary folders and in to a Managed Folder hierarchy. More info on MRM can be found here: http://technet.microsoft.com/en-us/library/bb123507.aspx My mailbox is now segmented so all of my email from the last year is in my personal mailbox folders (Inbox, Sent Items etc..); and all of my email that is older than one year is stored in the year based folder hierarchy underneath "Managed Folders\Research & Development". Step 2: Configure Outlook 2007 to not sync the "archive folders" Now that my mailbox is segmented, I want to configure Outlook 2007 (Cached mode) to only sync the email for the past year. I rarely access email that is older than 1 year (a couple times a month at most), so it is a worthwhile trade off to sync only recent email to my OST and thus improve the client experience. If I need to access email that is older than a year, I either bring up Outlook 2007 in Online Mode or use Outlook Web Access (OWA). Since accessing +1year old email is so rare, I have yet to hit a scenario where I was on the road and disconnected from a network and needed to access an old email. If it was a critical situation and I needed to access that email immediately, I could always access it via my Smart Phone and Exchange Active Sync. Configuring Outlook 2007 to not sync the "archive folders" is straightforward:

1. In Outlook 2007 (Exchange Cached Mode), right click the folder you want to set the sync policy on and choose Properties (e.g. Managed Folders\Research and Development\2006)

2. Click the Synchronization tab, and then click Filter.

3. Click the More Choices tab.

4. Set the Size (kilobytes) to "less than" 0. This is a simple way of configuring Outlook to not sync anything for this folder.

5. Click OK

6. If you already synced items to this folder and want to clean them out of the OST, then you need to go back to the Folder Properties General tab and click Clear Offline Items. This will expunge already synced items from the folder based on your new policy. I usually perform these steps on a newly created OST (to prevent the "archive folder" syncs occurring in the first place).

7. Follow steps 1-6 for every "archive folder" you do not want to sync to your OST. I recommend using a just a few archive folder for this reason.

8. Follow steps 1-7 every time you create a new Outlook Cached Mode profile on any computer. I followed these steps for my office workstation, my home computer and my laptop.

Step 3: Enjoy the results I have been very happy with Outlook 2007 performance after segmenting my mailbox and only sync'ing a year's worth of kept email (~2GB). This approach has allowed me to meet all of my criteria:

• Large Mailbox (several years worth of email stored in the Exchange Store)
  • I have access to several years worth of email from any device
• No PST usage
  • Since my mailbox is so large, I do not need PST's. PST's are a security/reliability risk as well as being a pain to manage and use. Life is much simpler with all of my email in one place, my mailbox (accessible via all of the standard clients). Keeping all of my email in my Exchange mailbox is the best/safest place to store it. It is stored on RAID10 storage, replicated via CCR, and is backed up on a regular basis. Before my large mailbox, my PST's were stored on non-raided storage, not replicated and backed up maybe once or twice a year.
• Good laptop Outlook 2007 cached mode user experience
  • By segmenting my mailbox in to two buckets (less than a year old and over a year old) and configuring Outlook 2007 folder sync policies, I have dramatically improved the client performance; even on my 5.4k laptop disk.
• Good workstation/home computer cached mode user experience
  • Outlook 2007 cache mode performance screams on my workstation and home computers (again, thanks to mailbox segmentation and Outlook 2007 folder sync policies).
• The ability to search my entire mailbox
  • For those few times a month I need to access email that is older than a year, I just do a quick search in OWA or open up Outlook in online mode.
• My email processing does not over burden the server
  • By running all of my clients in cached mode, I reduce my personal impact on the server (IO and CPU processing), by offloading some of the mail processing work to my client. The server costs do increase as the mailbox size increases when running in Online Mode. Running in cached mode solves this.

I have found that a large mailbox, combined with a few Outlook configuration tweaks can make for a great user experience. I have the best of both worlds; my Outlook experience is snappy, and I have access to all of my email from any standard Microsoft client. Till Outlook improves the OST scalability further, this is the configuration I live with on a daily basis. I look forward to the day where I can keep full copies of my mailbox synced to my client; but for now, I am satisfied using this "creative approach". To most of you, a 7.4GB mailbox is pretty big (huge even). I have worked with several customers recently who still have 25MB max mailbox sizes (I would fill it up with my profile in 2.5 days!). I have also worked with customers who have unlimited mailbox sizes and have users with mailboxes in the 20GB -> 60GB range! I have not tested this selective sync folder approach to 60GB mailboxes, but in principle it should work. It has got to be better than the 60GB OST experience! - Matt Gossage