Errors Retrieving File Shares on Windows Failover Cluster

Hi AskCore, Chinmoy here again. In today’s blog, I would like to share one more scenario in continuation to my previous blog on Unable to add file shares in Windows 2012 R2 Failover Cluster.

This is about WinRm a setting that could lead to failure on adding file shares using Windows 2012/2012R2 Failover Cluster Manager.

Consider a two-node Windows Server 2012 R2 Failover Cluster using shared disks to host a File Server role. To access the shares, we click on the file shares and go to the shares tab at the bottom.  We see the error on the Information column next to the Roles:

“There were errors retrieving the file shares.” clip_image002

There can be multiple reasons why Failover Cluster Manager would throw these errors. We will be covering one of the scenarios caused because of a WinRm configuration.

Scenario 1:

We cannot add new shares using Failover Cluster Manager, but can via PowerShell.  This may occur, if Winrm is not correctly configured.  WinRm is the Microsoft implementation of the WS-Management protocol and more can be found here.

If we have Winrm configuration issues, we may even fail to connect to remote servers or other Cluster nodes using Server Manager as shown below.

clip_image004

The equivalent PowerShell cmdlet reports the below error: -

PS X:\> Enter-PSSession Hostname
Enter-PSSession : Connecting to remote server hostname failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig". For more information, see the about_Remote_Troubleshooting Help topic.

At line:1 char:1
+ Enter-PSSession hostname
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (hostname:String) [Enter-PSSession], PSRemotingTransportException
+ FullyQualifiedErrorId : CreateRemoteRunspaceFailed

The above is a sign of WinRm being unable to connect to the remote server.

Let’s dig more, and check the event logs:

Log Name: Microsoft-Windows-FileServices-ServerManager-EventProvider/Operational
Event ID: 0
Source: Microsoft-Windows-FileServices-ServerManager-EventProvider
Description: Exception: Caught exception Microsoft.Management.Infrastructure.CimException: The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig".

The above event states that there is a communication issue with WinRm component. A quick way to configure WinRm is to run the command:

winrm quickconfig

This command starts the WinRM service and sets the service startup type to Auto-start. It also configures a listener for the ports that send and receive WS-Management protocol messages using either HTTP or HTTPS on any IP address. If it returns the following message:

WinRM service is already running on this machine.
WinRM is already set up for remote management on this computer.

Then try running the below command:

winrm id -r:ComputerName

You may receive the following message if WinRM is not able to communicate to the WinRS client. It also means we cannot resolve the destination using a loopback IP configured to "IP Listen List" for HTTP communications.

clip_image006

You can validate if the Loopback adapter IP is configured to "IP Listen List" for HTTP communication:

clip_image007

For this problem, run the below command:

clip_image008

On removing the loopback IP, we shall be able to add the file share successfully using the Failover Cluster console.

Scenario 2:

If we see below event:

Log Name:      Microsoft-Windows-FileServices-ServerManager-EventProvider/Operational
Source:        Microsoft-Windows-FileServices-ServerManager-EventProvider
Level:         Error
Description:
Exception: Caught exception Microsoft.Management.Infrastructure.CimException: The WinRM client received an HTTP status code of 504 from the remote WS-Management service.
   at Microsoft.Management.Infrastructure.Internal.Operations.CimSyncEnumeratorBase`1.MoveNext()
   at Microsoft.FileServer.Management.Plugin.Services.FSCimSession.PerformQuery(String cimNamespace, String queryString)
   at Microsoft.FileServer.Management.Plugin.Services.ClusterEnumerator.RetrieveClusterConnections(ComputerName serverName, ClusterMemberTypes memberTypeToQuery)

In the description of the event, you see that it is reporting a status code of 504.  What does this code mean?

Symbolic Name:       HTTP_STATUS_GATEWAY_TIMEOUT
Error description:       timed out waiting for gateway

It’s important to understand that we cannot have Failover Cluster Manager show all component's information(i.e. RPC, WMI, etc) that gets utilized in the background while enumerating the file shares.  Therefore, we need to depend on the event logs.

Resolution:

There are two solutions here:

  • If you do not normally use a Proxy Server for the cluster and don't need to access the internet, then follow these steps:

Try the below command to validate if you have any proxy set:

netsh winhttp show proxy

The output may look similar to the below with an IP Address listed against the proxy server(s):

Current WinHTTP proxy settings:

    Proxy Server(s) :  136.105.214.3:3128
    Bypass List     :  <local>

If you have a value set for proxy server(s), but don't need a proxy server set, then run the below command to fix the issue:

netsh winttp reset proxy

Later, after running the above command to reset proxy, on doing “netsh winhttp show proxy” you shall see as given below:

netsh winhttp show proxy

Current WinHTTP proxy settings:

Direct access (no proxy server).

  • If you do normally use a Proxy Server, then we will want to add your domain to the bypass list to keep the proxy server from causing problems

Try the below command to see what proxy server you have set:

netsh winhttp show proxy

The output may look similar to the below with an IP Address listed against the proxy server(s):

Current WinHTTP proxy settings:

    Proxy Server(s) :  136.105.214.3:3128
    Bypass List     :  <local>

As you can see, we don't have anything in the Bypass List, so let's add our local domain to the bypass (we are using Contoso as an example)

netsh winhttp set proxy proxy-server="136.105.214.3:3128" bypass-list="<local>;*contoso*"

Now let's look at the proxy output again:

netsh winhttp show proxy

We should see an output like below:

Current WinHTTP proxy settings:

    Proxy Server(s) :  136.105.214.3:3128
    Bypass List     :  <local>;*contoso*

 

Hope it helps to fix the issue. Good Luck!

Chinmoy Joshi
Support Escalation Engineer