Update: A study of Exchange 2007 SP1 Hub throughput with different message sizes


Background


This is a follow up to my recent post: A study of Exchange 2007 SP1 Hub throughput with different message sizes . Please refer to this post for full background information and test details. I promised in that post that I would add results for smaller message sizes and include data for SATA disks -- what we can call commodity storage'.

Columns 1 and 2 of the Table below show the results of 2 more scenarios, using same high end SCSI storage as in the previous post, adding message sizes of 57 KB and 80KB to the posted measurements. Columns 3 and 4 were taken on a newer, more powerful server but with less expensive 'commodity storage' (SATA drives).

Server Configuration


1. Ultra SCSI storage server: 2 processors x 2 core, 2.2 GHz, 800 MHz FSB, 1MB L2 cache per processor, 4 GB RAM, 400 MHz memory, Ultra 3 SCSI disk controller ("entry level") with 128 MB Write-Back Cache, 3 x Ultra320 Universal SCSI 15K RPM disk.

Optimized E2K7 transport database queue configuration:



  • 1 disk for DB logs
  • 1 disk for DB queue file
  • 1 disk for OS and other transport logs: Message Tracking, Connectivity, Agent logs, etc.

2. SATA disks server: 2 processors x 2 core, 2.33 GHz, 1.33 GHz FSB, 4MB L2 cache per processor, 16 GB RAM, 333 MHz memory, SATA interface without Battery Backed Write-Back Cache, 4 SATA disks 7200 RPM .


Optimized E2K7 transport database queue configuration:


  • 1 disk for DB logs
  • 2 disk for DB queue file, Raid 0 configuration
  • 1 disk for OS and other transport logs: Message Tracking, Connectivity, Agent logs, etc.
  • Disabled: Enable advanced performance disk policy
  • Disabled: Enabling write caching to disk disk policy

 





















































































































Hub Storage


Ultra SCSI storage


SATA disks


Limiting Resource


CPUBound


CPUBound


IOBound


IOBound


Message Size


57KB


80KB


57KB


80KB


SMTP Receive Throughput (msg/sec)


138.17


118.87


48.85


40.36


Aggregate Queue length (MAX)


358


409


44


64


Queue size in MB (MAX)


20.41


32.72


2.51


5.12


%CPU


84.60


84.15


17.65


17.81


Msg Cost (MCyc/msg)


50.93


59.29


30.36


37.30


Msg Cost (MCyc/ByteOfMsg)


893.51


741.13


532.63


466.25


Disk Writes/sec (log)


146.14


165.08


48.50


51.51


Disk Writes/sec (queue)


91.15


124.78


75.00


116.00


Disk Writes/msg (log)


1.06


1.39


0.99


1.28


Disk Writes/msg (queue)


0.66


1.05


1.54


2.87


Avg msec/write (log)


0


0


11


13


Avg msec/write (queue)


0


0


97


109


Avg  disk Queue length (log)


0.14


0.17


0.64


0.74


Avg  disk Queue length (queue)


0.18


0.33


3.90


6.57


Disk Reads/sec (log)


0.00


0.00


0.00


0.00


Disk reads/sec(queue)


0.00


0.00


0.00


0.00


Analysis of Results


The first 2 columns are an extension of the previous results, using the default 128 MB transport DB cache size.

Quoting the previous blog post: "storage is key for transport performance, all the above data only applies to a Hub server with at least an "entry level" SCSI controller with 128 MB of BBWC (battery backed write-back cache) that optimizes the IO pattern transport performs on steady state flow: continuous writes with very few or no reads."

The last 2 columns present the results on the new hardware. Notice the high disk latencies and the appearance of a disk queue without having a BBWC (battery backed write-back cache). Also notice the smaller MCyc/msg cost, this is because the new machine's cycles are much more powerful than the cycles on the older machines thanks to higher FSB, and more L2 cache.

New disclaimer: On the SATA disk machines, the Write Caching disk policies checkmarks Enable advanced performance and Enable advanced performance have been both disabled.


Yes, I tested briefly with these policies enabled.


Setting Enable advanced performance, raises the SATA disks performance to the level of the SCSI storage, resulting in even better throughput in the test because it's a faster machine! But it is not safe to enable it on production machines, because without a real hardware BBWBC controller data can be lost during hardware failures. See Windows Confidential - The Power of Bugs (April 2007 issue of TechNet Magazine) for a through explanation.

Elías Kaplan
SDET II, Exchange Shared






Share this post :

















Comments (3)
  1. pesospesos says:

    Perhaps a better test would be to use a server that can handle both SAS and SATA drives with the same BB controller.

    My exchange server is a poweredge 2950 with the perc6i SAS/SATA controller (256 MB cache).  I run SAS drives but I mixed arrays of SAS and SATA on my other 2950s (file server, DPM server, etc) and they run great.  It would be interesting to see the performance delta where only the drives are different…

  2. Elias says:

    Hi,

    Thanks for your comment.

    We don’t have the server with the mixed storage on the lab.

    Probably you can try a perf test?, Only if the server is not in production.

    The results should be similar to the ones published: with BBWC controller write latency is 0, injection throughput much higher.

    I did the test on the new server with "Enabled Advance Performance" on, using SATA disks, which  quite mimics a BBWC.

    But didn’t think that adding the results to the table was needed.

    Elias

  3. Drew says:

    I would love to see a similar test with different types of RAID (1,5,10) and different drives.

    I run a few servers and have RAID 1 for the OS, and RAID 5 for the logs and DBs. I am leaning to moving my logs to my OS drive.

    In the future I am trying to go to RAID 10 for the DBs and RAID 1 for the Logs with another RAID 1 for the OS.

    I like RAID 5 better cause of the ability to do online expansion.

    Your thoughts?

    -Drew

Comments are closed.

Skip to main content