SharePoint 2013 On-Prem AntiVirus exceptions and IP Offloading

Support has been seeing a spike in strange issues that have no direct ULS logs associated with the issues such as indexer performance or indexers out of sync or crawler performance slowdowns. I would like to just put some information out there for the first steps to review on your Search farm if you are seeing these types of issues.

 

#1 AntiVirus exceptions:

AV does a number on big blobs of data like indexes and lots of small little files being written and deleted quickly such as crawler temp directories.

Confirm that AntiVirus exceptions for the entire Search farm as well as the crawl component temp directories.

This is the list of directories and processes that should be excluded:

  • C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\Data (and sub directories)
  • C:\Program Files\Microsoft Office Servers\15.0\Data
  • C:\Program Files\Microsoft Office Servers\15.0\Bin
  • C:\Program Files\Microsoft Office Servers\15.0\Search\
  • C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\noderunner.exe (NodeRunner process exclusion)
  • C:\Users\[Search Service Account]\AppData\Local\Temp
  • If you Index data -RootDirectory is customized separately from SharePoint data directory, also exclude (this is where the Index lives)

 You could also just exclude the entire Office 15 hive if that is easier for your AV software.

Some more info is on TechNet but it isn't aimed at Search

https://support.microsoft.com/kb/952167/en-us

#2 IP Offloading:

TCP/IP (Or TOE or Chimney) was a big problem for FS4SP customers where it would cause IP Sec breakage resulting in entire communications outages between nodes. With SP2013 Search there have been some recorded communications performance issues and it is recommended to turn this off for IPV4 and IPV6 via:

 

netsh int ipv4 show global | findstr "Task Offload"

netsh int ipv6 show global | findstr "Task Offload"

 

There are also some more in depth instructions for nic cards with more options to ensure that IP Offloading is completely off. The instructions below are for the FAST Search product but hold true for SP2013 as well.

 https://support.microsoft.com/kb/2570111

 

For some search bait here is a single ULS log that has shown up multiple times for customers with symptoms above:

 

failed to get default collection for application SSA System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: Object reference not set to an instance of an object. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.NullReferenceException: Object reference not set to an instance of an object.  

 at Microsoft.Office.Server.Search.Administration.SearchApi.RunOnServer[T](CodeToRun`1 remoteCode, CodeToRun`1 localCode, Boolean useCurrentSecurityContext, Int32 versionIn)   

 at Microsoft.Office.Server.Search.Administration.SearchApi.RunOnServer[T](CodeToRun`1 remoteCode, CodeToRun`1 localCode)   

 at SyncInvokeGetVersion(Object , Object[] , Object[] )   

 at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)   

 at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)   

 at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)   

 at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)   

 at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)). StackTrace:

 at Microsoft.Office.Server.Native.dll: (sig=678c0f87-966f-4d99-9c94-b49e788d2672|2|microsoft.office.server.native.pdb, offset=131CE)

 at Microsoft.Office.Server.Native.dll: (offset=21BE5)

netsh int ipv4 show global | findstr "Task Offload"

netsh int ipv6 show global | findstr "Task Offload"