SharePoint Search 2010: Crawl status returns 503 Server Unavailable

I ran into an interesting problem where accessing the 2010 Search Service Application administration page returns a 503 Server unavailable for the Crawl Status.

image

 

Usually, a 503 indicates that the application pool is stopped that is hosting a site.  In this case, it’s the SharePoint Web Services Virtual directory hosting the Search Admin web service.  I selected the virtual directory, right clicked, and chose Advanced Settings and the GUID for the application pool is present.

image

 

Accessing the Application Pool shows that it is indeed stopped.

image

 

 

Cause 1

Try starting the application pool and refreshing the page.  If it still fails and the application pool is stopped again proceed to cause 2.

Cause 2

Usually,  this problem occurs because the application pool identity doesn’t have appropriate permissions in SQL.   For the service account running Search Service Application, you’ll need to ensure this account has the following roles on SQL Server.

db_owner and public roles on the following:

  1. Search Crawl Store Database
  2. Search Service Application Database
  3. Search Property Store Database

WSS_Content_Application_Pools role on the following

  1. SharePoint_AdminContentDatabase
  2. SharePoint_Config Database

 

Cause 3

The Search Service Account is missing from the local WSS_WPG group.  Access computer management, local users and groups, and ensure the Search Service Account is a member of this group.

image

 

 

Cause 4

This is a rare scenario but entirely possible.   If you check the ULS logs you may see something like the following when rendering the Search Admin page:

03/11/2013 21:46:10.42    w3wp.exe (0x1694)    0x08DC    SharePoint Server Search    Administration    dn26    High    System.Net.WebException: The remote server returned an error: (503) Server Unavailable.     at Microsoft.Office.Server.Search.Administration.SearchApi.RunOnServer[T](CodeToRun`1 remoteCode, CodeToRun`1 localCode, Boolean useCurrentSecurityContext, Int32 versionIn)     at Microsoft.Office.Server.Search.Administration.SearchApi..ctor(SearchServiceApplication application)     at Microsoft.Office.Server.Search.Administration.SearchServiceApplication.get_SearchApi()     at Microsoft.Office.Server.Search.WebControls.SearchApplicationSystemStatus.GetStatusString(SearchServiceApplication searchApp)     at Microsoft.Office.Server.Search.WebControls.SearchApplicationSystemStatus.BindContentToTable(Int32 pageNumber, Table contentTable)    208aa6ae-52e4-48d6-8168-414f56bb56fd

 

This can happen if the IIS_IUSRS group is missing from the local security policy.  Specifically, the group needs to be added to the following policies:

Impersonate a client after authentication and Log on as a batch job

 

Check and Fix:

  1. Go to Start, All Programs, Administrative Tools, and select Local Security Policy.
  2. Expand Local Policies, User Rights Assignment and check Log on as a batch job and Impersonate a client after authentication
  3. If IIS_IUSRS group is missing proceed to step 4.  If it’s present this is not the issue and disregard.
  4. Right click each policy go to Properties and add IIS_IUSRS
  5. Reboot or Perform a GP Update

Note:  If your unable to edit the local policy, you’ll need to contact your AD Administrators to update the policy.

It should look like the following after completion:

image

 

 

Resources:

https://technet.microsoft.com/en-us/library/cc678863(v=office.14).aspx#Section3

https://blogs.msdn.com/b/russmax/archive/2010/01/08/changing-sharepoint-2010-service-accounts.aspx

 

Thanks,

Russ Maxwell, MSFT