Some of the fun we have in product support is that, once a new product is released nowadays, we get to navigate the uncharted waters of new security settings interoperating with our customers’ real world environments.
With Windows XP and Server 2003 we saw that there were challenges brought about by the SMB signing, and LMCompatibility level security settings. SMB Signing is a way of guaranteeing the originator of the traffic since it is signed by that node. LMCompatibility, put simply, is a way of telling your computer to not use less than a certain version of NTLM authentication since older versions are less secure.
Both of these are good things from a security perspective. Frankly, if they are disabled or lessened, then your systems are less secure.
But in the real world there are plenty of people who have old computers (Windows 9x, NT) that may not be compliant with enhanced security. These types of things are may not always be disseminated well at a product’s release and we are forced to play catch up. If you saw my post regarding TCP Auto-Tuning a few months ago then you’ve heard this tune before.
LM Compatibility level is a way of setting your Windows computer to use only a specified level of LanMan authentication (NTLM). This is done via a registry value which is noticed by the LSA at boot:
Why am I bothering to post about this “old hat” stuff? Well,
Please check if the below symptom happens based on our support experience:
– Your problem occurs only when connecting to the resource from a Vista client, but may not occur from other operating systems
-You do not specify a custom LMcompatibliltylevel setting in any of your group policies
-Doesn’t necessarily have to be a network file sharing device back end…but more likely to be.
-Network traffic will appear similar to that below (SMB negotiation details excepted for brevity):
No. Time Source Destination Protocol Info
152 07:48:17.447552 22.214.171.124 126.96.36.199 SMB Negotiate Protocol Request
153 07:48:17.449232 188.8.131.52 184.108.40.206 SMB Negotiate Protocol Response
164 07:48:17.458380 220.127.116.11 18.104.22.168 SMB Session Setup AndX Request, User: DOMAINuser1; Tree Connect AndX, Path: \LOCALFILESVRIPC$
182 07:48:20.410098 22.214.171.124 126.96.36.199 SMB Session Setup AndX Response, Error: STATUS_LOGON_FAILURE
How can you work around this behavior? Well, the best way would be to bring the same minimum level of security to all devices involved. This can be a difficult thing to do when you are stuck with inherited infrastructure and a limited budget.
If you can’t have all devices meet that minimum security then you will be forced to allow less secure authentication in order to get your business flowing. To do that, lower the LMcompatibliltylevel to a lower number that the other device can handle. Then reboot for that to take effect.
Here’s a link article that goes into good detail about NTLM authentication and what the LMcompatibliltylevel setting does: