Do you have a sleepy NIC?

I continue to run into this issue over and over in the field so I wanted people to be aware of this possible problem. In a Database Availability Group (DAG), if your databases are randomly mounting or flipping from one server to another, for no apparent reason (including across datacenters) you may be suffering from your network interface card (NIC) going to sleep. And that’s not a good thing.

Power Management on the NIC

In the power management settings for the NIC on Windows Server, make sure you are not allowing the NIC to go into power save mode. Why is this important? It seems like at least once a month I’ve run into customers who have this power management setting turned on and more than one of them even had it turned on for their replication network. They were seeing some odd behavior – for example, their databases randomly flipping from one DAG node to another for no apparent reason. And yes, they were all on physical machines.

Here are the steps to look at this configuration: use Device Manager to change the power management settings for a network adapter.

To disable all Power Management settings in Device Manager, expand Network Adapters, right-click the adapter > Properties > Power Management, and then clear the Allow the computer to turn off this device to save power check box.

Screenshot: Network adapter properties | Power Management tab
Figure 1: Disable power management for the network adapter from the Power Management tab

Some of your network adapters may not have the Power Management tab available. This is a good thing, as your NIC is not able to go to sleep. This means there is one less item to worry about in your setup!

CAUTION Be careful when you change this setting. If it’s enabled and you decide to disable it, you must plan for this modification as it will likely interrupt network traffic. It may seem odd that by just making a seemingly non-impacting change that the NIC will reset itself, but it definitely can. Trust me; I had a customer ‘test’ this during the day by accident… oops!

PowerShell to the rescue

In addition, now that PowerShell is able to be used for just about everything, there is this page that has a PS script available to make this change. There are additional links and related forum threads to review with supplementary information near the bottom of the script download page.

This modifying script will run against all physical adapters to the machines you deploy it to, and you can also modify the script to disable wireless NIC’s. With PS, don’t forget that you can use this script to blast down these changes to all of your Exchange servers with a single step.

GPO and regedit

For those of you that are more comfortable with regedit and creating GPO’s to help control these settings, that option is also available. This page has information on both ‘one off’ fixes that you can download a .reg file and manually deploy, or using GPO Preferences, you can edit the values in a GPO and apply those changes to an Exchange Server OU (Organizational Unit).

The one step to note with the regedit process is which NIC you are working with and how many NIC’s your server has. The registry only knows of the first, second, third, etc. number of NIC’s. Now if you have identical builds between all of your servers, then this option certainly will ensure that all current and any future servers placed into an OU with the GPO applied will adhere to the proper registry settings.

Also don’t forget, you can record all of your changes on a Windows Server 2008R2 or higher OS, by using the Problem Steps Recorder (PSR) tool.

There you have it: if your DAG Databases are randomly becoming active from one server to another with no apparent reason, you may have a sleepy NIC. Please confirm that you have avoided this setting as you build out not only your DAG environment, but all Exchange related servers. Thank you.

Mike O’Neill

Comments (28)
  1. Anonymous says:

    This is a key item to check while deploying Windows Servers for any collaboration platforms. Great article, Mike.

  2. Anonymous says:

    Great artical but, it didn’t work in my environment. I came to the same conclusion Steve did before scrolling down… We unchecked the NIC power save mode on all Exchange 2010 servers, rebooted each server and 5 hrs later our DAG failed over again.

  3. Mike_O'Neill says:

    @Anonymous. There are 3 things I always check when I roll into a customer with a DAG.

    1. Turn off sleepy NIC’s.
    2. Ensure there is an IP set for the DAG in each subnet/AD Site where DAG members live. Occasionally, engineers just put one in and not an IP for the secondary site, which is typically another subnet range.
    3. Ensure your 2010 DAG network is collapsed:

    Any one of these could cause inconsistent DAG behavior. Hope that helps.

  4. Mike_O'Neill says:

    With the help of Lalita Jat, another PFE team member, we have an additional answer about the ‘high performance’ setting within the Windows Server OS:

    • If the server is set to High Performance – Windows places the system in the highest performance state and disables the dynamic scaling of performance in response to varying workload levels. Therefore, special care should be taken before setting the power plan to High Performance as this can increase power consumption unnecessarily when the system is underutilized.
    • We will have to disable “Allow the computer to turn off this device to save power” option of Power management from NIC. Setting the server in High Performance would not stop the NIC to go in sleep mode whenever there is no activity, as that is a setting at an OS level. The setting on NIC will take preference in this situation.

    Hopefully this answers the question about the high performance setting.

  5. Good catch Mike, one of the most important reason of DAG Wars!

  6. Samir Marwaha says:

    Mike does it needs to be don on the CAS/HT servers as well?

  7. Jon says:

    If the box is checked, is there any log that would show the computer turning off the NIC to save power?

  8. Heath Graves says:

    I think this might be my gremlin.  Kudos if this works.

  9. I have a similiar issue with one customer, but in my case just few databases randomly become active from one server to another. I could note yesterday that just 2 DBs were changed. If it is related to the power management all DBs should be moved, right?

  10. Mike_O'Neill says:

    @ Samir Marwaha: While I have seen this DAG Wars issue occur on Mailbox servers, it would be advisable to ensure all servers (not only Exchange ones) have this setting disabled. When would I want a server NIC to go to sleep? Never would be my guess.

    @ Heath Graves: This is one of the first 3 things I check rolling onsite. NIC settings, DAG IP’s for each subnet that has a DAG member, and collapse of the DAG network. Any or all 3 can cause gremlin issues on randomly activating DB’s in a DAG.

    @ Tiago Ferreira de Souza: I have not dug into specifics if all or just some DB’s are impacted. If the NIC only drops a few packets, then wakes back up, I could see that only some DB’s could be impacted and not necessarily the entire server with all DB’s

    Again, ensure that all 3 steps are covered if you are having random DB’s mounting for what appears to be no reason at all.

    1. Disable NIC sleep mode

    2. Ensure the DAG network is collapsed.…/exchange-2010-collapsing-dag-networks.aspx

    3. Ensure that there is an IP assigned to the DAG where each DAG member lives in that subnet:…/dd351172(v=exchg.150).aspx

    Hope that helps.

  11. Raji Subramanian says:

    Great Article…The sleepy NIC can be applied to all Microsoft application server as best practice…

  12. Thank you Mike. In my case is missing only disable NIC sleep mode.

  13. Priit says:

    I always untick that on all my NIC-s. Have been wondering for years why they are ticked on all NIC-s by default on Windows Servers as that can only cause issues and lead to server not being available. I suspect an oversight on the MS Windows Server team as it appears to be the case no matter what drivers or make of NIC you have.

  14. Well, is this the default on a Server 2012 box? If yes, then sorry to say MS, you can do better! ;-)

    GOOD catch indeed, Mike!

  15. Sam Tudorov says:

    It should also be noted that the boxes on the power management tab can easily get check again during server maintenance, e.g. when you or your operations staff upgrade the driver to a newer release.  :-)

    I have always unchecked these options on every server I've ever worked on.  I don't see why it always defaults to being checked or even why these options need to exist at all for a server class machine or OS.

  16. Oli H. says:

    Also, please have a look to your general power saving settings:

    Degraded overall performance on Windows Server 2008 R2…/2207548

    Poor virtual machine application performance may be caused by processor power management settings…/

    And don't forget to check the host BIOS power settings, too.

    keyword: C-States

  17. Joshua Bines says:

    Hi Mike,

    We are investigating the risk around this issue, there seems to be a lack information around what will cause a NIC with this setting enabled to go into sleep mode? I'm assuming either a lack of traffic across the interface or something else? Any info would be appreciated.


  18. Scott says:

    I agree with Josh.  I don't have these issues in Exchange, but I have been having issues with a client NIC that has been disconnecting periodically throughout the day.  I ultimately suspect a bad driver, but for kicks, I tried this setting.  Sure enough, no issues so far — knock on wood.

  19. Steve says:

    According to this article (…/2740020), it only effects the NIC when the computer goes into sleep. Why would any server be going into a sleep state?

  20. sr195 says:

    Fully agree with Steve. Why should I do this when running a Server in "High Performance" Mode. So the Server operates always in S0 state!? and never goes to sleep. Something I like cause it's a Server ^^ and not a Client. The details of my NIC also say that in S1 (which is sleep) the NIC goes to Energy Saving D3. Conclusion: No S1 -> No D3 of the NIC

  21. Mike_O'Neill says:

    I have read the MS articles several times and still do not fully understand why a NIC in server production environments, including DAG's, would ever go to sleep. However, I see it monthly with customers having their DAG DB's activate randomly for no apparent reason. They stop the random behavior after we implement this change. Something somewhere must be triggering it; different drivers, different vendors, different patches…

    Would love to find the end all solution to this issue.

  22. The PowerShell script says you need to reboot the server for it to take affect? Is this true or can it be done on the "fly" knowing that it will cause a reset of the NIC.

  23. Joshua Bines says:

    Thanks for the heads up Mike, We don't have any issues here but it won't hurt too turn it off anyway.

    I'd be interested to know if all these cases are with physical servers?


  24. Mike_O'Neill says:

    @ Joshua Bines: I actually only found it on virtual servers at first (running on VMWare), then, ran into more setups and found it on any/all types of servers. I think it depends on the driver, as I have seen several (and others pointed out) NIC's don't even have the 'power management' tab. Which is ideal. I don't know of a compiled list of which drivers/versions have/do not have the power management tab.

    @ Will Shepherd: I doubt it would require a reboot, as the setting takes effect upon making the change. You are correct, the NIC will reset itself. Unfortunately, I can't test this, as my hyper-v server NIC's do not have the power management tab available. Maybe someone can verify if they use the PS that it does not require a reboot? I just don't have current access to a machine that I can test.

  25. @ Mike_O’Neill – The permanent fix would probably be a GPO/DSC (PowerShell 4.0’s Desired State Configuration) setting that tells all NICs on the host to disable Power Saving; then you set that GPO on your Server Group(s).

    Here’s the Official Method to doing it, under ‘Fix It Myself’ it lists the Registry entry – there is one per-device, so one would need a way to enumerate all devices under that:…/2740020

    Here is someone who made a script to do it (his is set for Intel NICs only, but could be modified fairly easily):…/243073-we-need-a-gpo-or-script-to-adjust-power-management-on-nics

  26. Rajeev Ujjwal says:

    Mike, Appreciated for highlighting this issue –

    But, if this is enabled by default at OS level then this is wrong as per my view, MS should disbale this by default in next rollup or SP release …I know power saving is also important but it should not create this type of unnecessary issue in the email service of organization..

  27. Terrance Brennan says:

    Excellent article!  Changing that setting stopped the constant flipping of my DAG DBs.  Mine were all ESX VMs but apparently it happens with physicals as well.  Definitely a must for an Exchange server build.

  28. Mike_O'Neill says:

    Thank you Terrance for the feedback, glad it helped your situation. And thank you for helping me pin point this error.

    Also, I'm getting other feedback from engineers and it appears that (most/all?) HP NIC's do not have the power management tab available. Which is a good thing.

Comments are closed.