Hybrid Search: Inbound Query fails with 401

Article
05/06/2015

Last year I came across an issue where Inbound query was failing consistently.

Issue applies to: April 2014 Cumulative Update or a later cumulative update forr SharePoint 2013 on-premises farm.

When doing an Inbound Search ( i.e You search from SPO and expect to get results from SPO and Onprem) you see the query fails with below error message:

1¾System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.GetResponse() at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute() at Microsoft.SharePoint.Client.ClientContext.GetFormDigestInfoPrivate() at Microsoft.SharePoint.Client.ClientContext.EnsureFormDigest() at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery() at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.RetrieveDataFromRemoteServer(Object unused) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.ProcessRecordCore(IRecord record)

To investigate further I took a look at the Onprem and SPO side of logs: If you are interested in the stack trace look below or can skip to solution section :)

Logs from OnPrem:

08/07/2014 13:39:07.48 w3wp.exe 0x08A4 SharePoint Portal Server User Profiles ae0sx Unexpected Error trying to search in the UPA. The exception message is 'System.ArgumentException: Exception of type 'System.ArgumentException' was thrown. Parameter name: value at Microsoft.SharePoint.Administration.Claims.SPIdentityProviders.GetIdentityProviderType(String value) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.SearchUsingNameIdOrThrow(UserProfileManager upManager, String nameId, String nameIdIssuer) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetSingleUserProfileFromClaimsList(UserProfileManager upManager, IEnumerable`1 identityClaims)' a53bac9c-90c4-0075-b18e-d8c02dfc8e41

08/07/2014 13:39:07.48 w3wp.exe 0x08A4 SharePoint Portal Server User Profiles ae0su High The set of claims could not be mapped to a single user identity. Exception Exception of type 'System.ArgumentException' was thrown. Parameter name: value has occured. a53bac9c-90c4-0075-b18e-d8c02dfc8e41

08/07/2014 13:39:07.48 w3wp.exe 0x08A4 SharePoint Foundation Claims Authentication ae0tc High The registered mappered failed to resolve to one identity claim. Exception: System.InvalidOperationException: Exception of type 'System.ArgumentException' was thrown. Parameter name: value ---> System.ArgumentException: Exception of type 'System.ArgumentException' was thrown. Parameter name: value at Microsoft.SharePoint.Administration.Claims.SPIdentityProviders.GetIdentityProviderType(String value) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.SearchUsingNameIdOrThrow(UserProfileManager upManager, String nameId, String nameIdIssuer) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetSingleUserProfileFromClaimsList(UserProfileManager upManager, IEnumerable`1 identityClaims) --- End of inner exception stack trace --- at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetSingleUserProfileFromClaimsList(UserProfileManager upManager, IEnumerable`1 identityClaims) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.<>c__DisplayClass2.<GetMappedIdentityClaim>b__0() at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass5.<RunWithElevatedPrivileges>b__3() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetMappedIdentityClaim(Uri context, IEnumerable`1 identityClaims) at Microsoft.SharePoint.IdentityModel.SPIdentityClaimMapperOperations.GetClaimFromExternalMapper(Uri contextUri, List`1 claims) a53bac9c-90c4-0075-b18e-d8c02dfc8e41

08/07/2014 13:39:07.48 w3wp.exe 0x08A4 SharePoint Foundation Claims Authentication af3zp Unexpected STS Call Claims Saml: Problem getting output claims identity. Exception: 'System.InvalidOperationException: Exception of type 'System.ArgumentException' was thrown. Parameter name: value ---> System.ArgumentException: Exception of type 'System.ArgumentException' was thrown. Parameter name: value at Microsoft.SharePoint.Administration.Claims.SPIdentityProviders.GetIdentityProviderType(String value) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.SearchUsingNameIdOrThrow(UserProfileManager upManager, String nameId, String nameIdIssuer) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetSingleUserProfileFromClaimsList(UserProfileManager upManager, IEnumerable`1 identityClaims) --- End of inner exception stack trace --- at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetSingleUserProfileFromClaimsList(UserProfileManager upManager, IEnumerable`1 identityClaims) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.<>c__DisplayClass2.<GetMappedIdentityClaim>b__0() at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass5.<RunWithElevatedPrivileges>b__3() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode) at Microsoft.Office.Server.Security.UserProfileIdentityClaimMapper.GetMappedIdentityClaim(Uri context, IEnumerable`1 identityClaims) at Microsoft.SharePoint.IdentityModel.SPIdentityClaimMapperOperations.GetClaimFromExternalMapper(Uri contextUri, List`1 claims) at Microsoft.SharePoint.IdentityModel.SPIdentityClaimMapperOperations.ResolveUserIdentityClaim(Uri contextUri, ClaimCollection inputClaims) at Microsoft.SharePoint.IdentityModel.SPIdentityClaimMapperOperations.GetIdentityClaim(Uri contextUri, ClaimCollection inputClaims, SPCallingIdentityType callerType) at Microsoft.SharePoint.IdentityModel.SPSecurityTokenService.GetLogonIdentityClaim(SPRequestInfo requestInfo, IClaimsIdentity inputIdentity, IClaimsIdentity outputIdentity, SPCallingIdentityType callerType) at Microsoft.SharePoint.IdentityModel.SPSecurityTokenService.EnsureSharePointClaims(SPRequestInfo requestInfo, IClaimsIdentity outputIdentity, SPCallingIdentityType callerType) at Microsoft.SharePoint.IdentityModel.SPSecurityTokenService.AugmentOutputIdentityForRequest(SPRequestInfo requestInfo, IClaimsIdentity outputIdentity) at Microsoft.SharePoint.IdentityModel.SPSecurityTokenService.GetOutputClaimsIdentity(IClaimsPrincipal principal, RequestSecurityToken request, Scope scope)'. a53bac9c-90c4-0075-b18e-d8c02dfc8e41

Logs from SPO: ( Currently only MS can pull these logs)

RemoteSharepointProducerSystem.Net.WebException: The remote server returned an error: (401) Unauthorized.

at System.Net.HttpWebRequest.GetResponse()

at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()

at Microsoft.SharePoint.Client.ClientContext.GetFormDigestInfoPrivate()

at Microsoft.SharePoint.Client.ClientContext.EnsureFormDigest()

at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()

at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.RetrieveDataFromRemoteServer(Object unused)

at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.ProcessRecordCore(IRecord record)

ExecuteFlowInternal Flow:Microsoft.RemoteSharepointFlow EvaluationException: Microsoft.Office.Server.Search.Query.RemoteSharepointException: 1<=>System.Net.WebException: The remote server returned an error: (401) Unauthorized.

at System.Net.HttpWebRequest.GetResponse()

at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()

at Microsoft.SharePoint.Client.ClientContext.GetFormDigestInfoPrivate()

at Microsoft.SharePoint.Client.ClientContext.EnsureFormDigest()

at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()

at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.RetrieveDataFromRemoteServer(Object unused)

at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

at Microsoft.Office.Server.Search.RemoteSharepoint.RemoteSharepointEvaluator.RemoteSharepointProducer.ProcessRecordCore(IRecord record)

at Microsoft.Ceres.Evaluation.Processing.Producers.ProducerRecordSetSink.Put(IRecord record)

at Microsoft.Ceres.Evaluation.Engine.ErrorHandling.HandleExceptionRecordSetSink.DoWithoutTryCatch(IRecord record)

at Microsoft.Ceres.Evaluation.Processing.Producers.ProducerRecordSetSink.PushToOutput()

at Microsoft.Ceres.Evaluation.Processing.Producers.ProducerRecordSetSink.Put(IRecord record)

at Microsoft.Ceres.Evaluation.Engine.Aborting.AbortableRecordSetSink.Put(IRecord record)

at Microsoft.Ceres.Evaluation.Engine.ErrorHandling.HandleExceptionRecordSetSink.DoWithTryCatch(IRecord record)

Solution:

 $config = Get-SPSecurityTokenServiceConfig

$config.AuthenticationPipelineClaimMappingRules.AddIdentityProviderNameMappingRule("OrgId Rule", [Microsoft.SharePoint.Administration.Claims.SPIdentityProviderTypes]::Forms, "membership", "urn:federation:microsoftonline")

$config.Update()

Do an IISRESET after this.

Note: Steps need to be executed only on any one SP onprem server.

Hybrid Search: Inbound Query fails with 401

Additional resources