Troubleshooting Distributed Cache for SharePoint 2013 On Premise


Here is a script that will give you a good starting point for troubleshooting Distributed Cache for SharePoint 2013 On Premises.

Make sure to save it as a .ps1 file and run the script (Administrator) on a running Cache Host like "PS C:\> .\DistributedCacheScript.ps1", or right click "Run with PowerShell" as it will simply not work with copy paste.

I will keep adding and changing functionality if time permits, please feel free to suggest any improvements.

Small post on how to interpret the output you can find here:

http://blogs.technet.com/b/filipbosmans/archive/2015/09/07/how-to-check-for-issues-with-distributed-cache-and-the-script.aspx

Recommendations post:

http://blogs.technet.com/b/filipbosmans/archive/2015/10/17/sharepoint-distributed-cache-recommendations.aspx

Tools:

http://blogs.technet.com/b/filipbosmans/archive/2016/02/29/tools-to-troubleshoot-distributed-cache-sharepoint-2013.aspx

Slightly adjusted 2016 version:

https://blogs.technet.microsoft.com/filipbosmans/2016/06/01/troubleshooting-distributed-cache-for-sharepoint-2016-on-premises/

 

It will:

  • Create a folder @ C:\Distributed Cache [ServerName]
  • Create files which contain things like local event logs for DC, DC commands, exported configuration XML for the Cache Cluster etc.

It will not make any changes what so ever anywhere.

Added DCCheck_No_Logs.ps1 file at the bottom, contains no Application nor AppFabric logs.

Uncomment to get full output.

Apologies for the excessive formatting code added, I like to keep things nice and clean 🙂

Decided to add more than just Cluster/Server checks, now includes also output to verify SharePoint permission issues for instance.

Due to adding more things lately, you might want to run it on all Cache Hosts when you suspect non-Cluster related settings being related to your issue.

Firewall check will only work on Server 2012 or higher.

 Provided As Is

========================

Write-Host "Please make sure to run this script on a Cache Host, as an Administrator, and always via saving it first in a .ps1 script, so do not copy paste in console" -foregroundcolor red -backgroundcolor yellow
Write-Host ""
Write-Host "This script will:" -foregroundcolor red -backgroundcolor yellow
Write-Host ""
Write-Host "Create a folder C:\Distributed_Cache ServerName" -foregroundcolor red -backgroundcolor yellow
Write-Host "Takes Application and AppFabric logs if needed" -foregroundcolor red -backgroundcolor yellow
Write-Host "Runs some commands to get more information about the cluster and hosts" -foregroundcolor red -backgroundcolor yellow
Write-Host "Start-Transcript for any troubleshooting issues running commands" -foregroundcolor red -backgroundcolor yellow
Write-Host ""

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue
$ComputerName = gc env:computername
New-Item -Name "Distributed_Cache $ComputerName" -ItemType Directory -Path C:\
$Folder = Get-Item "C:\Distributed_Cache $ComputerName"

$GetCache = "Get-Cache"
$GetCacheHost = "Get-CacheHost"
$GetCacheClusterHealth = "Get-CacheClusterHealth"
$GetSPServiceInstance = "Get-SPServiceInstance"
$GetSPServiceInstanceDetailed = "Get-SPServiceInstance Detailed"
$CacheStatistics = "Cache Statistics"
$GetWinEvent = "AppFabric Logs"
$GetApplicationLogs = "Application Logs"
$GetContent = "AppFabric Configuration File"
$GetVersion = "AppFabric Version"
$GetService = "AppFabric Service"
$GetRegistry = "Registry"
$NetStat = "Netstat"
$NetSh = "NetShell"
$FireWallRules = "FireWall Rules"
$ClientSetting = "Cache Settings"
$DCClientSetting = "Get-SPDistributedCacheClientSetting -ContainerType "
$MemoryAndManufacturer = "Memory and Manufacturer"
$LocalRunningServices = "Local running Services"
$GetCacheAllowedClientAccounts = "Accounts + Local Groups"
$AppFabricProcess = "Memory used by AppFabric Process"
$TimerJobs = "Timer Jobs"
$Hosts = "Hosts File"
$NSLookup = "DNS"
$FarmBuild = "SharePoint Build"
$NIC = "Network Card"
$Line = "=================="

Start-Transcript -Path $Folder\Transcript.txt
Use-CacheCluster
<# Set-CacheLogging -LogLevel Verbose -Path $Folder\PSLogging.txt
$Line | Out-File $Folder\GetInfo.txt #>
$Line | Out-File $Folder\GetInfo.txt -Append
$GetCacheClusterHealth | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-CacheClusterHealth | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$GetCache | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-Cache | % {Get-AFCacheConfiguration -CacheName $_.CacheName} | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$ClientSetting = "Cache Settings" | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
$Array = ("DistributedLogonTokenCache", "DistributedViewStateCache", "DistributedAccessCache", "DistributedActivityFeedCache", "DistributedActivityFeedLMTCache", "DistributedBouncerCache", "DistributedDefaultCache", "DistributedSearchCache", "DistributedSecurityTrimmingCache", "DistributedServerToAppServerAccessTokenCache")
foreach($CacheDetail in $Array)
{
$CacheDetail | Out-File $Folder\GetInfo.txt -Append
Invoke-Expression ("$DCClientSetting" + "$CacheDetail") | Out-File $Folder\GetInfo.txt -Append
}
$Line | Out-File $Folder\GetInfo.txt -Append
$CacheStatistics | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-Cache | % {
$CacheName = $_.CacheName
Get-CacheStatistics -CacheName $CacheName | Add-Member -MemberType NoteProperty -Name 'CacheName' -Value $CacheName -PassThru | Out-File $Folder\GetInfo.txt -Append
}
$Line | Out-File $Folder\GetInfo.txt -Append
$GetCacheHost | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-CacheHost | fl | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$GetSPServiceInstance | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-SPServiceInstance | ? {($_.service.tostring()) -eq "SPDistributedCacheService Name=AppFabricCachingService"} | Format-List -Property TypeName,Status,Id,Server | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$GetSPServiceInstanceDetailed | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-SPServiceInstance | ? {($_.service.tostring()) -eq "SPDistributedCacheService Name=AppFabricCachingService"} | fl | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$NetStat | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Netstat | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$Netsh | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
netsh interface ipv4 show tcpconnections | Out-File $Folder\GetInfo.txt -Append
netsh interface ipv4 show tcpstats | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$Hosts | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-Content "C:\Windows\System32\drivers\etc\hosts" | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$NSLookup | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
nslookup $ComputerName | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$TimerJobs | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
(Get-SPTimerJob -Type "Microsoft.Office.Server.UserProfiles.LMTRepopulationJob").HistoryEntries | ft -Property Status,StartTime,EndTime,ErrorMessage | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$NIC | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-NetIPAddress -AddressFamily IPv4 | Select-Object InterfaceAlias, IPv4Address | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$FarmBuild | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
(Get-SPFarm).buildversion | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$MemoryAndManufacturer | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
$TotalPhysicalMemory = (gwmi Win32_computersystem).TotalPhysicalMemory/1MB
("$TotalPhysicalMemory" + " MB Total Physical Memory") | Out-File $Folder\GetInfo.txt -Append
$TotalFreeMemory = (gwmi -Class win32_OperatingSystem).FreePhysicalMemory/1kb
("$TotalFreeMemory" + " MB Total Free Memory") | Out-File $Folder\GetInfo.txt -Append
$ManufacturerSystem = (gwmi Win32_computersystem).Manufacturer
("$ManufacturerSystem" + " System Manufacturer") | Out-File $Folder\GetInfo.txt -Append
$ManufacturerBIOS = (gwmi Win32_BIOS).Manufacturer
("$ManufacturerBIOS" + " BIOS Manufacturer") | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$LocalRunningServices | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-SPServiceInstance -Server $ComputerName | where {$_.Status -eq "Online"} | select TypeName | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$GetCacheAllowedClientAccounts | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
Get-CacheAllowedClientAccounts | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
net localgroup WSS_WPG | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
net localgroup WSS_ADMIN_WPG | Out-File $Folder\GetInfo.txt -Append
Write-output `n | Out-File $Folder\GetInfo.txt -Append
net localgroup Administrators | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$AppFabricProcess | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-Process -Name DistributedCacheService | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
$FireWallRules | Out-File $Folder\GetInfo.txt -Append
$Line | Out-File $Folder\GetInfo.txt -Append
Get-NetFirewallRule –DisplayGroup "Remote Service Management" | Out-File $Folder\GetInfo.txt -Append
Get-NetFirewallRule –DisplayGroup "File and Printer Sharing" -Description "Echo Request messages are sent as ping requests to other nodes." | Out-File $Folder\GetInfo.txt -Append
Export-CacheClusterConfig -Path $Folder\ExportCacheClusterConfig.txt
<# $Line | Out-File $Folder\AppFabricLogs.txt
$GetWinEvent | Out-File $Folder\AppFabricLogs.txt -Append
$Line | Out-File $Folder\AppFabricLogs.txt -Append
Get-WinEvent -ProviderName "Microsoft-Windows Server AppFabric Caching" | fl | Out-File $Folder\AppFabricLogs.txt -Append
$Line | Out-File $Folder\ApplicationLogs.txt
$GetApplicationLogs | Out-File $Folder\ApplicationLogs.txt -Append
$Line | Out-File $Folder\ApplicationLogs.txt -Append
$GetCacheHostName = (Get-CacheHost).Hostname
foreach($Hostname in $GetCacheHostName)
{
Get-EventLog -ComputerName $HostName Application -EntryType Error,Warning | where {$_.Message -like "*Distributed*"} | fl -Property * | Out-File $Folder\ApplicationLogs.txt -Append
} #>
$GetVersion | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
(Get-ItemProperty "C:\Program Files\AppFabric 1.1 for Windows Server\PowershellModules\DistributedCacheConfiguration\Microsoft.ApplicationServer.Caching.Configuration.dll" -Name VersionInfo).VersionInfo.ProductVersion | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
$GetContent | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
Get-Content "C:\Program Files\AppFabric 1.1 for Windows Server\DistributedCacheService.exe.config" | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
$GetService | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
Get-WmiObject win32_service | Where-Object Name -eq AppFabricCachingService | fl name, startname, startmode, state | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
$GetRegistry | Out-File $Folder\AppFabric.txt -Append
$Line | Out-File $Folder\AppFabric.txt -Append
Get-ItemProperty "hklm:\SOFTWARE\Microsoft\AppFabric\V1.0\Configuration" | Out-File $Folder\AppFabric.txt -Append

Stop-Transcript

========================

Hope it helps.

Filip

========================

Edited 04/03/2015:

Commented out Set-CacheLogging, there is little use for it and sometimes it seems to hang at this point.

========================

Edited 08/09/2015

Removed useless logging and changed what it looks for.

Added AppFabric version to see which CU is installed.

Added Config file to check for Garbage Collection.

Added Registry key AppFabric to check if connectionstring is populated.

Added AppFabric Service to check if correct account is set and running.

========================

Edited 29/09/2015

Added netstat and netsh commands to see if and which ports are opened and listening and the number of open connections.

Added Cache Container output to check if the values are raised/changed.

Added Firewall rules check to see if Remote Management and ICMP is enabled.

Changed Get-SPServiceInstance output to make it easier to see if there is a mismatch between AppFabric and SharePoint.

Added DCCheck_No_Logs.ps1 file to download.

========================

Edited 05/10/2015

Added Local services to abide by MS recommendations.

Added Process, Total/Free Memory output to check for possible throttling and memory overconsumption/configuration issues.

Added Manufacturer/BIOS information in case it is unknown if the machine is a VM or not, to get an idea if Hyper-Visor "Virtual Memory" is used or not.

Added Accounts output and local groups to check for missing permissions.

========================

Edited 07/12/2015

Added Cache Statistics.

Added smaller things like dns/IP Address/nslookup/Farm build version.

DCCheck_No_Logs.ps1

Comments (4)

  1. BlueSky2010 says:

    Thank you for putting this together!

    Since you are open to suggestion (:p) - I would say if there were a scenario as to when you would use a script like this that might help others more.

  2. Hi, was planning to do this but never found the time, will update the post some time soon.
    You will be able to spot a lot of common issues and rule out cluster misconfiguration.

Skip to main content