Can I really use JBOD storage with Exchange 2010?

I just did an Exchange 2010 architectural design session for a university in Ohio and this was a key question they asked me.  The answer is yes, you truly can leverage JBOD (Just a Bunch Of Disks) storage with large TB SATA drives for Exchange 2010 now. Microsoft IT has deployed Exchange 2010 with JBOD storage and we are benchmarked as some of the heaviest mail profiles available. This proves using JBOD storage with Exchange 2010 can handle even the highest of mail loads/stress and IOPS per user (.11+).

 

Why would I consider JBOD storage for Exchange 2010?

Storage is going to be your largest expense in deploying Exchange 2010.  If you can reduce your storage costs by anywhere from 90%+ to 40%, maintain the same performance, and significantly grow your mail quota per user it becomes a no brainer. This is the same conclusion MS IT came to. They also did not need to grow their storage staff to maintain JBOD.

 

How is this possible?

Cheap SATA and JBOD storage are possible since, with Exchange 2010, the product team managed to significantly increase and optimize database performance which resulted in reducing the IOPS per user by another 70% over Exchange 2007. This translates to the use of cheaper storage can be leveraged for equivalent performance of higher cost storage used with Exchange 2007.

image

For example, a 7200RPM 1TB SATA drive will provide an estimated 75-80 IOPS per spindle. This now becomes a viable spindle type for Exchange 2010 with the additional 70% IOPS/mailbox reduction. If you add in our new DAG availability, where we replicate and failover at the database layer, JBOD becomes a viable option as well.

 

What do I have to have in place in order to use JBOD storage?

The recommended DAG architecture for JBOD storage is a minimum of 3 mailbox server nodes with 3 database copies per database. You dedicate one spindle per database and associated transaction logs. If a spindle fails, since it is non-RAIDed, it doesn’t matter since you have TWO other good copies of the database to failover to on two other dedicated spindles. 

What does a sample JBOD storage look like?

This is just a sample architecture for reference. You could use larger SATA drives now such as 2TB SATA drives to lower your spindle count. If you look closely you will see, 3 copies of any given database spread across 3 servers.

image

Are there tools I can use to validate my JBOD storage?

Yes, first thing I would recommend would be to use the Mailbox Calculator to level set that JBOD is a good fit for your organization. Note: Make sure you specify 3 or more database copies or the calculator will not recommend JBOD as a viable option. The nice part about the calculator is you can play with different SATA spindles (7200 RPM, 3.5”, 1TB, 2TB, etc) to arrive at different results.

Sample Mailbox calculator data showing JBOD is a fit:

image

Once you have purchased JBOD storage, I would leverage the following stress tools:

JetStress

LoadGen

to benchmark your storage and hardware. Grab those tools here.

Is there a storage matrix I can use to reference?

I found this useful storage matrix put together from the product team:

Exchange 2010 Storage Guidance

Stand Alone

Database Availability Group: 2 nodes, 2 Database copies

Database Availability Group: 3+ nodes, 3+ Database copies

Storage Type

Direct Attached Storage (DAS)

Supported

Supported

Supported

Storage Area Network (SAN): iSCSI

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications.

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications.

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications.

Storage Area Network (SAN): Fibre Channel (FC)

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications.

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications. Best Practice = Do not place both database copies on the same physical spindles.

Supported. Best Practice = Do not share physical disks backing Exchange data with other applications. Best Practice = Do not place both database copies on the same physical spindles.

Network Attached Storage (NAS): SMB

Not Supported

Not Supported

Not Supported

Physical Disk Type

SATA

Supported, requires battery backed caching array controller for data integrity

Supported, requires battery backed caching array controller for data integrity

Supported, requires battery backed caching array controller for data integrity

SAS

Supported

Supported

Supported

FC/FATA

Supported

Supported

Supported

SSD (Flash Disk)

Supported

Supported

Supported

Physical Disk Write Caching (enabled)

Not Supported

Not Supported

Not Supported

Storage RAID

RAID recommended

RAID recommended

RAID optional

EDB Volume

RAID5/6, RAID10, RAID1

RAID5/6, RAID10, RAID1

JBOD, RAID5/6, RAID10, RAID1

Log Volume

RAID1, RAID10

RAID1, RAID10

JBOD, RAID1, RAID10

Disk Array RAID Stripe Size (kb)

256KB

256KB

256KB

Storage Array Cache Settings

75% Write Cache, 25% Read Cache (with Battery Backed Cache)

75% Write Cache, 25% Read Cache (with Battery Backed Cache)

75% Write Cache, 25% Read Cache (with Battery Backed Cache)

Database/Log file placement

     

Database/Log Isolation

Best Practice (for recoverability) = separate database file (.edb) and logs from same Database on to different volumes backed by different physical disks

Database file (.edb) and logs from same Database can share same volume and same physical disk.

Database file (.edb) and logs from same Database can share same volume and same physical disk. This is a best practice for JBOD/RAID'less storage scenario where one or more volumes store the edb and log files backed by the same physical disk.

Database Files/Volume

Based on backup methodology

Based on backup methodology

RAID = based on backup methodology, JBOD = one DB file/volume is recommended

Log Streams/Volume

Based on backup methodology

Based on backup methodology

RAID = based on backup methodology, JBOD = one log stream/volume is recommended

Windows Disk Type

     

Basic Disk

Recommended

Recommended

Recommended

Dynamic Disk

Supported

Supported

Supported

Partition Type

     

GUID Partition Table (GPT)

Recommended

Recommended

Recommended

Master Boot Record (MBR)

Supported

Supported

Supported

Partition Alignment

Windows 2008 Default: 1MB

Windows 2008 Default: 1MB

Windows 2008 Default: 1MB

Volume Path

Drive Letter or Mount Point (mount point host volume must be RAIDed)

Drive Letter or Mount Point (mount point host volume must be RAIDed)

Drive Letter or Mount Point (mount point host volume must be RAIDed)

File System

NTFS support only

NTFS support only

NTFS support only

NTFS Defragmentation

Not required, not recommended

Not required, not recommended

Not required, not recommended

NTFS Allocation Unit Size

64KB for both edb and log volumes

64KB for both edb and log volumes

64KB for both edb and log volumes

NTFS Compression

Not Supported for Exchange Database files

Not Supported for Exchange Database files

Not Supported for Exchange Database files

NTFS Encrypted File System (EFS)

Not Supported for Exchange Database files

Not Supported for Exchange Database files

Not Supported for Exchange Database files

Windows Bitlocker (volume encryption)

Supported for all Exchange database and log files

Supported for all Exchange database and log files

Supported for all Exchange database and log files

 

More on storage planning here and here.

DAG samples here.