One of the most common calls we get is that file transfers are “slow,” causing us to collect network traces and perfmon logs in order to determine if the transfer is slower than it should be and, if so, what is the cause. I had a case recently where a little-known and rarely-experienced change to Windows Server 2003 Service Pack 2 caused some pain for a customer.
The customer had a cluster of Server 2003 Terminal Servers, which then accessed a cluster of file servers on the backend. The customer noticed that after they upgraded some of their Terminal Servers to Service Pack 2, things had significantly “slowed down” as a result. We did some standard troubleshooting and made changes to things like the Scalable Networking Pack, but it didn’t seem to make a difference. A process that used to take 6-7 seconds was now taking 25 seconds to perform. The strangest thing about it was that regular file copies from the file servers were fine. The slowdown occurred when the users logged in and a configuration file was read from the file server line-by-line. They had a script that reproduced the issue consistently by accessing a file and doing that line-by-line transfer. We took network traces and saw the following:
Service Pack 1
In the above transfer, you can see that the SMB buffer in the Read AndX Request starts at 4096 bytes, but quickly increases to 32K, resulting in the fast response. However, when we looked at the post SP2 server, it looked different:
Service Pack 2
This time, the buffer didn’t increase beyond 4K throughout the transfer, even though during the Session Setup AndX Request, we said our Max Buffer size was 16K. We found it especially strange that full file copies didn’t have this same behavior.
Based on that information, I did some research and found out that from Service Pack 1 to Service Pack 2, we disabled a ReadAhead feature since hotfix 894463 is included in SP2. (Please note that while hotfix 894463 makes the change, KB328237 discusses ReadAhead and provides the registry settings to re-enable the feature).
- 894463 – Programs that open multiple files may experience longer delays between each file opening on a Microsoft Windows XP-based computer or on a Microsoft Windows 2000-based computer – http://support.microsoft.com/default.aspx?scid=kb;EN-US;894463
- 328237 – Some programs do not work as expected when large files are opened – http://support.microsoft.com/default.aspx?scid=kb;EN-US;328237
This ReadAhead feature doesn’t benefit all file transfers, but does impact performance with this small, line-by-line transfer and sometimes when Office files are opened over the network. While the issue appeared on a Windows Server 2003 box in my case, it can also affect other clients (by client, I mean the computer initiating the file transfer) such as Windows XP and Windows 2000. In order to return to their previous performance, on the Terminal Server we added the ReadAheadGranularity DWORD referenced in 328237 and set it to 8 for eight pages to read ahead (equaling 32 KB). Once the customer rebooted their server, the issue was resolved and their rollout to SP2 could continue.
Here’s some other helpful Information for tweaks to improve file transfers:
- 320829 – How to modify the default SizReqBuf value in Windows 2000 and Windows Server 2003 – http://support.microsoft.com/default.aspx?scid=kb;EN-US;320829
- 296264 – Configuring opportunistic locking in Windows – http://support.microsoft.com/default.aspx?scid=kb;EN-US;296264
Thanks and happy copying!