Stop hurting yourself by: Not updating the drivers and firmware in Windows and Windows Server.

Article
03/20/2018

Applies to:

Windows 10 1803 (tbd, codename RS4)

Windows 10 1709 (Fall Creators update, codename RS3)

Windows 10 1703 (Creators update, codename RS2)

Windows 10 1607 (Anniversary update, codename RS1) / Windows Server 2016 (codename RS1)

Windows 10 1511 (November update, codename TH2)

Windows 10 1507 (RTM, codename TH1)

Windows 8.1/Windows 2012 R2

Windows 8/Windows 2012

Windows 7 SP1/Windows 2008 R2 SP1

Windows Vista/Windows 2008

As a Premier Field Engineer (PFE), I get to go on-site and from time to time, I get to troubleshoot Windows and Windows Servers that are having reliability and performance problems.

Q: When should you update your driver(s) and firmware?

When a new piece of hardware you've installed doesn't work automatically

When a hardware is generating an error, like a Device Manager error code

if you're troubleshooting a problem with the device

if a driver update enables new features you'd like to utilize.
- Windows client: (e.g. video cards and sound cards.)
- Windows Server: (e.g. NIC, Storage/HBA)

maybe before/after upgrading to a new version of Windows.

a security flaw in the driver(s)/firmware

The most common scenario is the following:

"We update the driver(s) and firmware when the image of Windows and/or Windows Server are built."

Q: How often do you guys update the driver(s) and firmware?

A: We never do once the machine is imaged.

Q: How long do you guys keep your hardware in production?

A: 3-6 years or longer.

Q: What if you are having a Bug Check (a.k.a. Stop Error, Blue Screen of Death (BSOD), and for you *nix IT admin's Kernel Panic), the easiest thing to do what?

Option 1: Update the driver(s)/firmware and/or
Option 2: Update the security products and/or
Option 3: Get a kernel memory dump and analyze it.

A: It's Option 1. Update the driver(s)/firmware.

If you want to get root cause, setup for a kernel and/or complete memory dump (or Active memory dump on Windows Server 2016):

Coming soon: How to generate a kernel or a complete memory dump file in Windows Server 2012 and Windows Server 2012 R2

https://blogs.technet.microsoft.com/yongrhee/2015/04/05/coming-soon-how-to-generate-a-kernel-or-a-complete-memory-dump-file-in-windows-server-2012-and-windows-server-2012-r2/

Scenario 1: An I.T. administrator was troubleshooting a Print Server related issue, where the Windows clients were having problems connecting to the \\PrintServer\PrinterShare. The issue was with a 3rd party network .dll that hooked into the SMB (CIFS) components which was fixed via an update to their Virtualization drivers (equivalent to a Hyper-V Integration tools).

Scenario 2: An I.T. administrator had a Windows Server, for troubleshooting purposes he disabled the on-board NIC's. But when he went to re-enable the on-board NIC's, it wouldn't turn on. This specific Server hardware manufacturer had an issue with their firmware. The hardware manufacturer had to replace the motherboard. There was a firmware update that had been out for 6+ months that would have prevented this issue from occurring.

Scenario 3: A Windows Server was dropping network packets. The fix was to update the NIC driver where the problem would no longer occur.

Scenario 4: A Windows Server was dropping buffers (nonpaged pool kernel memory) when we were trying to collect a "Storport ETW trace" while using the "Windows Performance Recorder UI (WPRUI))" trace of their slow disk i/o on SAN attached volumes. We updated the SCSI controller and the buffer dropping went away.

Scenario 5: An I.T. shop had replaced their Windows clients with all SSD drives. There was a caveat, it had shipped with an issue where if it had written/read x MB of data, it would 'brick' the SSD. The fix was in a firmware update.

Scenario 6: A SSD firmware update would let you go from 250 MB/sec reads to 500 MB/sec reads. Yes, just a firmware update on the SSD. Why wouldn't you take advantage of it?

Scenario 7: A Windows client was observing random erroneous behavior, once we updated the Storage Controller driver, the symptoms went away.

Etc…

Q: How often should we look for a driver(s)/firmware update?

A: At a minimum every (1) year (12 months).

Q: What’s the difference between inbox drivers and OEM drivers?

A: The inbox drivers that ship with Windows tend to have the basic functionality for the device.

e.g. If you want to duplicate or extend your screen, you need to make sure that the OEM video driver is installed.

e.g. 2 Some NIC card vendors have ‘templates’ (best practices) and if you want to apply a Network template based on a specific workload (IIS/Virtualization/File server) you need to have the OEM NIC driver installed.

e.g. 3 If you have a Boot from SAN based Windows Server, you want to make sure that you are using the OEM (3rd party) DSM/HBA drivers and firmware, and their Multi-pathing software.

Q: What about the drivers from “Windows Update”?

A: We provide drivers for special circumstances, such as when the OEM doesn’t support a particular device model any longer. If the OEM hardware vendor provides a ‘newer’ version of the driver or firmware, you should install the OEM version.

Q: What if my 3rd party OEM hasn’t updated the driver/firmware hosted on their website, but the 3rd party OEM hardware manufacturer has a newer version?

A: You probably know that your 3rd party OEM vendor doesn’t support those (3rd party OEM hardware manufacturer) drivers. But, if you have a test environment, you could test the newer driver from the 3rd party manufacturer and see if the issue goes away. If it does, and you are still under warranty and/or support, please open a case with your 3rd party OEM vendor.

Q: So how do we update the driver(s)/firmware on our environment?

A: There are many methods, please check with your hardware manufacturer.

Here are a few examples:

Friday Support: How to update your drivers and software

https://blogs.technet.microsoft.com/smallbusiness/2013/03/08/friday-support-how-to-update-your-drivers-and-software/

Updating Firmware for Disk Drives in Windows Server 2016 (TP4)

https://blogs.technet.microsoft.com/filecab/2016/01/25/updating-firmware-for-disk-drives-in-windows-server-2016-tp4/

Keeping Surface Firmware Updated with Configuration Manager

https://blogs.technet.microsoft.com/thejoncallahan/2016/06/20/keeping-surface-firmware-updated-with-configuration-manager/

Q: How do we roll it out?

A: In the same phased approach.

First, test it on a Test/QA environment.
Second, test it on the IT admin's (if they have the hardware)
Third, test it 'non-mission critical' systems.
Fourth, do a segment of the hardware at a time.

And by the way, for the folks that want a 'root cause', if it's a "Windows Client or Windows Server", and it's really 'mission critical', why wait until the problem 'reoccurs'.

Usually the immediate task in hand is restoring service.
The 2nd task is, a plan to collect data to determine why it happened and what can you do to prevent the issue from reoccurring in the future.

Thanks,

Yong (In Dallas, TX. for a couple of days)

P.S. Other “Stop hurting yourself by” posts:

Stop hurting yourself by: Not applying the non-security updates for Windows and Windows Server.

https://blogs.technet.microsoft.com/yongrhee/2018/03/12/stop-hurting-yourself-by-not-applying-the-non-security-updates-for-windows-and-windows-server/

Stop hurting yourself by: Disabling IPv6, why do you really do it?

https://blogs.technet.microsoft.com/yongrhee/2018/02/28/stop-hurting-yourself-by-disabling-ipv6-why-do-you-really-do-it-2/

WMI: Stop hurting yourself by using “for /f %%s in (‘dir /s /b *.mof *.mfl’) do mofcomp %%s”

https://blogs.technet.microsoft.com/yongrhee/2016/06/23/wmi-stop-hurting-yourself-by-using-for-f-s-in-dir-s-b-mof-mfl-do-mofcomp-s/

More information:

Windows 10 – Audio Troubleshooting Tips

https://blogs.technet.microsoft.com/mediaq/2015/07/30/windows-10-audio-troubleshooting-tips/