Update: A study of Exchange 2007 SP1 Hub throughput with different message sizes

[The_Exchange_Team]

Background

This is a follow up to my recent post: A study of Exchange 2007 SP1 Hub throughput with different message sizes . Please refer to this post for full background information and test details. I promised in that post that I would add results for smaller message sizes and include data for SATA disks -- what we can call commodity storage'. Columns 1 and 2 of the Table below show the results of 2 more scenarios, using same high end SCSI storage as in the previous post, adding message sizes of 57 KB and 80KB to the posted measurements. Columns 3 and 4 were taken on a newer, more powerful server but with less expensive 'commodity storage' (SATA drives). Server Configuration 1. Ultra SCSI storage server: 2 processors x 2 core, 2.2 GHz, 800 MHz FSB, 1MB L2 cache per processor, 4 GB RAM, 400 MHz memory, Ultra 3 SCSI disk controller ("entry level") with 128 MB Write-Back Cache, 3 x Ultra320 Universal SCSI 15K RPM disk. Optimized E2K7 transport database queue configuration:

• 1 disk for DB logs
• 1 disk for DB queue file
• 1 disk for OS and other transport logs: Message Tracking, Connectivity, Agent logs, etc.

2. SATA disks server: 2 processors x 2 core, 2.33 GHz, 1.33 GHz FSB, 4MB L2 cache per processor, 16 GB RAM, 333 MHz memory, SATA interface without Battery Backed Write-Back Cache, 4 SATA disks 7200 RPM . Optimized E2K7 transport database queue configuration:

• 1 disk for DB logs
• 2 disk for DB queue file, Raid 0 configuration
• 1 disk for OS and other transport logs: Message Tracking, Connectivity, Agent logs, etc.
• Disabled: Enable advanced performance disk policy
• Disabled: Enabling write caching to disk disk policy

 

Hub Storage	Ultra SCSI storage		SATA disks
Limiting Resource	CPUBound	CPUBound	IOBound	IOBound
Message Size	57KB	80KB	57KB	80KB
SMTP Receive Throughput (msg/sec)	138.17	118.87	48.85	40.36
Aggregate Queue length (MAX)	358	409	44	64
Queue size in MB (MAX)	20.41	32.72	2.51	5.12
%CPU	84.60	84.15	17.65	17.81
Msg Cost (MCyc/msg)	50.93	59.29	30.36	37.30
Msg Cost (MCyc/ByteOfMsg)	893.51	741.13	532.63	466.25
Disk Writes/sec (log)	146.14	165.08	48.50	51.51
Disk Writes/sec (queue)	91.15	124.78	75.00	116.00
Disk Writes/msg (log)	1.06	1.39	0.99	1.28
Disk Writes/msg (queue)	0.66	1.05	1.54	2.87
Avg msec/write (log)	0	0	11	13
Avg msec/write (queue)	0	0	97	109
Avg  disk Queue length (log)	0.14	0.17	0.64	0.74
Avg  disk Queue length (queue)	0.18	0.33	3.90	6.57
Disk Reads/sec (log)	0.00	0.00	0.00	0.00
Disk reads/sec(queue)	0.00	0.00	0.00	0.00

 

Analysis of Results

The first 2 columns are an extension of the previous results, using the default 128 MB transport DB cache size. Quoting the previous blog post: "storage is key for transport performance, all the above data only applies to a Hub server with at least an "entry level" SCSI controller with 128 MB of BBWC (battery backed write-back cache) that optimizes the IO pattern transport performs on steady state flow: continuous writes with very few or no reads." The last 2 columns present the results on the new hardware. Notice the high disk latencies and the appearance of a disk queue without having a BBWC (battery backed write-back cache). Also notice the smaller MCyc/msg cost, this is because the new machine's cycles are much more powerful than the cycles on the older machines thanks to higher FSB, and more L2 cache. New disclaimer: On the SATA disk machines, the Write Caching disk policies checkmarks Enable advanced performance and Enable advanced performance have been both disabled. Yes, I tested briefly with these policies enabled. Setting Enable advanced performance, raises the SATA disks performance to the level of the SCSI storage, resulting in even better throughput in the test because it's a faster machine! But it is not safe to enable it on production machines, because without a real hardware BBWBC controller data can be lost during hardware failures. See Windows Confidential - The Power of Bugs (April 2007 issue of TechNet Magazine) for a through explanation. Elías Kaplan SDET II, Exchange Shared