If you ever encounter some messages that takes time to be delivered from HUB servers to Mailbox servers (we call this step “Mapi Submission” or “MAPI Delivery”) or even from HUB to another HUB in a remote site, and your network don’t have bandwidth or latency or general reliability or other configuration issues such as wrong MTU, wrong speed set on equipments, then check for the Windows TCP Chimney and RSS settings.
These are part of what we call the “Network Scalability Pack” (http://technet.microsoft.com/en-us/network/bb545631.aspx) and their primary use is to offload the CPU from processing network packets, offloading it down to the network card. It’s supposed to remove network processing load from the CPU and give the work to the network card instead.
In the past, when installed on Windows 2003 servers, we had issues with this setting being used with Exchange (see this : http://support.microsoft.com/kb/948496), and we recommended to deactivate these TCP chimney, RSS and sometimes network DMA features as well. But experience showed that we can encounter sporadic issues as well on Windows 2008/Exchange 2007 and 2010 (especially for HUB submission queues) under certain circumstances due to these settings; these circumstances can be the network card driver TCP offload features not very compatible with how Exchange uses the network, or simply the network card driver being too old to handle the TCP offloading features… hard to tell exactly why anyways until we do a full raw debugging, which I didn’t do.
Anyways, when we encountered network issues on Windows 2008/Exchange 2007/2010 and we also troubleshooted the issue thoroughly to ensure that were not related to bandwidth , latency , general reliability , other configuration issues such as wrong MTU, wrong speed set on equipments,bad network driver, etc…, disabling TCP chimney and RSS magically solved our issues; AND, no reboot is theorically necessary after changing these using netsh (but as we always say, when in doubt, reboot).
- To check for the current TCP Chimney, RSS and NetDMA settings do :
Netsh int tcp show global
For Windows 2008 R2 it will give you the following output-ish:
PS C:UsersAdministrator.CONTOSO> netsh int tcp show global
Querying active state…
TCP Global Parameters
Receive-Side Scaling State : enabled
Chimney Offload State : automatic
NetDMA State : enabled
Direct Cache Acess (DCA) : disabled
Receive Window Auto-Tuning Level : normal
Add-On Congestion Control Provider : ctcp
ECN Capability : disabled
RFC 1323 Timestamps : disabled
- To print out network cards that actually have the TCP Chimney functionnality activated :
Netsh int tcp show chimneystats
On the below example, you see Under the “Supp” column the value “No”, meaning that TCP Chimney is not supported or not enabled on the Network card driver settings, otherwise you’d see “Yes” with some values on “TMax” and “PMax”.
PS C:UsersAdministrator.CONTOSO> netsh int tcp show chimneystats
Idx Supp Atmpt TMax PMax Olc Failure Reason
— —- —– —- —- — ————–
11 No -n/a- 0 0 0 -n/a-
Idx – Interface (NIC) index used by the system
Supp – Interface (NIC) supports TCP chimney offload
Atmpt – System has attempted TCP connection offload
TMax – Offload capacity advertized by the NIC
PMax – Offload capacity observed by the system
Olc – Number of currently offloaded connections
FailureReason – Reason why last attempt to offload a connection failed
Use `netsh int tcp show chimneystats <Idx>‘ for more details.
- To disable TCP Chimney and RSS:
netsh int tcp set global chimney=disabled
netsh int tcp set global rss=disabled
For details and reference about the above commands, see http://support.microsoft.com/kb/951037