Keep your friends close, but your cluster node configuration closer. (Comparing differences across failover cluster nodes).

 

In my role, I often get to review failover clusters. Typically we look for configuration problems or deviations from best practices. One of the most challenging aspects of managing and maintaining a failover cluster is keeping the nodes identically configured. This means identical drivers, driver versions, service packs, hotfixes, services and applications across all cluster nodes.

Failover clusters are usually deployed with each node identically configured; however, as time passes the nodes tend to diverge. Drivers get updated on one node, but not the others. Hotfixes get installed on one node, but not the others. Your challenge, should you choose to accept it, is to keep the nodes the same. What you need are some tools to audit the configuration across the nodes, to look for differences.

 

How to compare cluster nodes in Windows Server 2008 or Windows Server 2008 R2

In Windows Server 2008 and Windows Server 2008 R2 there are built in capabilities to compare failover cluster nodes. The tool is called “Validate a Configuration Wizard”. Simply run the validation wizard and it will (among other things) compare cluster nodes for differences in BIOS, drivers and hotfixes. (Note that the cluster validation wizard is much more powerful than simply comparing the cluster nodes. If you administer (or plan to administer) 2008/R2 failover clusters you should get to know the validation wizard).

Validate a New or Existing Failover Cluster: http://technet.microsoft.com/en-us/library/cc772450.aspx

What do you do, though, if you want to compare Windows Server 2003 cluster nodes, or systems that aren’t cluster nodes?

 

Comparing Differences in Other Scenarios – Enter the PowerShell cmdlet: Compare-Object

(Note: All examples are executed from a machine with PowerShell v.2.0. However, the target machines are a mix of 2008 R2 and Windows Server 2003. It is NOT required to have Powershell/.Net/WinRM on the target machines. You must simply have administrative privileges on the target machines and WMI must be working properly)

If you can extract information from two systems, PowerShell can do the comparison for you. For example, use the Get-Hotfix cmdlet to pull hotfix information from two different computers, and then use the Compare-Object cmdlet to compare them.

Start by pulling the hotfix information from a remote computer “ParentDC1”.

Get-Hotfix –computer ParentDC1

It’s Never That Easy, is it?

Notice all the “File 1” garbage? Let’s get rid of that.

Get-Hotfix –computer ParentDC1 | where{$_.HotfixID –like “KB*”}

Now, let’s hold that data in a variable so we can use it to compare to another system.

$a = Get-Hotfix –computer ParentDC1 | where{$_.HotfixID –like “KB*”}

Then, do likewise to collect information from another computer “ChildDC2”.

$b = Get-Hotfix –computer ChildDC2 | where{$_.HotfixID –like “KB*”}

Now, simply compare the two collections with Compare-Object. Note the use of the –property switch so we only compare hotfixIDs (and not hostnames, installed dates or installed by).

Compare-Object $a $b –property HotfixID

To interpret the output from Compare-Object you note the direction of the SideIndicator. When we called Compare-Object we put $a (ParentDC1) on the left and $b (ChildDC2) on the right. Thus, if the SideIndicator points to the left, the hotfix appears on ParentDC1 (and not ChildDC2). If the SideIndicator points to the right, the hotfix appears on ChildDC2 (and not ParentDC1).

What if you also want to see the hotfixes installed on each node that are the same? Include the –IncludeEqual switch

Compare-Object $a $b –property HotfixID -IncludeEqual

Note on Get-Hotfix, from http://msdn.microsoft.com/en-us/library/windows/desktop/aa394391(v=vs.85).aspx. “Starting with Windows Vista, this class returns only the updates supplied by Component Based Servicing (CBS). These updates are not listed in the registry. Updates supplied by Microsoft Windows Installer (MSI) or the Windows update site (http://update.microsoft.com) are not returned by Win32_QuickFixEngineering.” This means get-hotfix may not return all installed hotfixes when run against Windows 7 or Windows Server 2008 R2. If you want to know more about the new component-based servicing model, see the following blog: http://blogs.technet.com/b/askperf/archive/2008/04/23/understanding-component-based-servicing.aspx.

That’s pretty cool and relatively easy. How about comparing services across systems? Same logic, but use the Get-Service cmdlet to get the data.

$a = Get-Service –computer ParentDC1
$b = Get-Service –computer ChildDC2
Compare-Object $a $b -property name

If you’d like to only consider services that are running, you can modify your commands, as follows:

$a = Get-Service –computer ParentDC1 | where {$_.status –eq “running”}
$b = Get-Service –computer ChildDC2 | where {$_.status –eq “running”}
Compare-Object $a $b -property name

How About Something Harder – Like Driver Differences?

Now things start getting a little interesting (or not, depending on your tastes). Unfortunately, there is no Get-Driver cmdlet. So we need to go into the weeds, and get some data using WMI. Basically, we combine two WMI queries. The first one, gets a list of installed drivers on a system.

$Query = “SELECT Name, PathName FROM Win32_SystemDriver WHERE PathName IS NOT NULL”
$driverList = Get-WmiObject –Query $Query –ComputerName ParentDC1
$driverlist

The second WMI query can take a pathname and return details about the driver (including version)

$Query2 = “SELECT Name, Version, Manufacturer, LastModified FROM CIM_DataFile WHERE Name = ‘C:\\Windows\\system32\\Drivers\\afd.sys”
$DriverInfo = Get-WMIObject –Query $Query2 –Computername ParentDC1
$DriverInfo

So you’ll need to create a collection of driver information on the first system by walking through the driver list. Then create a second collection of driver information for the second system by walking through its driver list. Then you use Compare-Object to compare the two collections. You could report differences in driver names, to report drivers installed on one system, but not the other. Or you could report differences in driver versions, to report differences in driver versions across systems.

If you’ve read this far, you’ve probably figured out that you’ll need to script the solution so you put in the necessary loops and create the custom data collections. I’ll spare you the details, and let you look through the attached script (drivers.ps1). To use the script you can either specify a single hostname and the script will report on driver information for that host.

.\Drivers.ps1 ParentDC1

Or, you can include two hostnames and the script will show you a difference in drivers installed on the systems.

.\Drivers.ps1 MNTools1 MNTools2

Or, you can include two hostnames with the –versions switch to only report differences in driver versions (a driver installed on both systems, of a different version).

.\Drivers.ps1 ParentDC1 ParentDC2 -versions

Finally, you could pass the script two hostnames with the –both switch to report both driver differences and driver version differences. Note that the script will report if no differences are found.

.\Drivers.ps1 ParentDC1 ChildDC2 –both

I hope you enjoy the script. Remember all the usual caveats apply to the script, so use it at your own risk. I hope you now have some tools to keep your cluster nodes more closely configured with respect to hotfixes, services and drivers.

Doug Symalla

 

Update 15.October.2012.  See Rick Bergman’s blog with an updated hotfix comparison tool.  This tool will allow you to compare hotfixes across more-than-two cluster nodes.

http://blogs.technet.com/b/askpfeplat/archive/2012/10/15/comparing-hotfixes-across-multiple-node-failover-clusters.aspx

Update (3.April.2013):  To centralize the storage of all AskPFEPlat scripts, we are now storing them on the TechNet Script Center Repository.  This specific script can be found at the following location:

http://gallery.technet.microsoft.com/scriptcenter/How-To-Compare-Drivers-78490789