Interesting problem when adding an ADFS Proxy

I am working on a blog post (step-by-step) for the Proxy component and I ran into a problem yesterday that ran me around pretty good.  We have seen this issue or variations of it on some support cases recently, so I thought the actual problem itself would make a good post.

The problem is caused by permissions to the private key on the Client Authentication Certificate needed.  In my initial attempt to setup and document the Proxy component, I made a request to my Standalone CA for a client authentication certificate.  After approving the request, the only option from the certificate web page was to "install this certificate".  Next, when I viewed the certificate snap-in on the proxy server, I noticed that the certificate was installed to the user store and not the computer store.  I simply did a copy paste operation from user to computer.  This appeared to work for me because when I double clicked the certificate, it looked fine.  I saw the "You have a private key" on the general tab and I assumed all was well.

When I went to test - I received a failure.  The first thing I did was run the ADFS Diagnostic tool.  I ran it on the FS-A, then copied the file to the FS-A Proxy.  I passed all tests and the tool was not finding the failure! 

Here are the Event Log and Debug Logs from my FS-A and FS-A Proxy when I attempted to access the application with the Proxy in place:

From the FS-A

Event Viewer:

Event Type: Error

Event Source: ADFS Federation Service

Event Category: None

Event ID: 664

Date: 6/3/2008

Time: 5:13:09 PM

User: N/A

Computer: ADFSACCOUNT

Description:

The Federation Service failed a privileged Web method call because Secure Sockets Layer (SSL) client authentication information was not available.

This event can occur if the client does not provide a client certificate or if Internet Information Services (IIS) rejects the client's certificate because it does not chain to a trusted root certification authority in the Federation Service.

User Action

If this is a valid call from the Federation Service Proxy, ensure that the root of the Federation Service Proxy client certificate is trusted by the Federation Service.

Debug logs:

2008-06-03T22:13:09 [INFO] Processing HTTP POST: https://adfsaccount.adatum.com/adfs/fs/FederationServerService.asmx

2008-06-03T22:13:09 [VERBOSE] Received message that is not SignIn Request or Response.

2008-06-03T22:13:09 [ERROR] MethodInvocationCheck: Client cert is not present

2008-06-03T22:13:09 [EVENTLOG] Error ProxyWebMethodAccessDeniedNoCert ()

2008-06-03T22:13:09 [ERROR] MethodInvocationCheck: Denying access

From the FS-A Proxy

Event Viewer:

Event Type: Error

Event Source: ADFS

Event Category: None

Event ID: 605

Date: 6/3/2008

Time: 5:13:09 PM

User: N/A

Computer: FSA-PROXY

Description:

The Federation Service Proxy encountered an exception when it called a Federation Service Web method.

Federation Server URL: https://adfsaccount.adatum.com/adfs/fs/FederationServerService.asmx

Web method: GetProxyTrustConfiguration

Proxy certificate thumbprint: ECF1FE79E51231DF48098E1044233FCBDABF04CC

This may cause a user request to fail.

User Action

The exception details may give an indication of the precise problem.

Check network connectivity between the Federation Service Proxy and the Federation Service.

Ensure that the Federation Service is running.

Ensure that the Federation Service Proxy client authentication certificate has been added to the list of proxy authentication certificates in the Federation Service trust policy.

Ensure that the Federation Service Proxy client authentication certificate chains to a root that is trusted by the Federation Service.

Ensure that the Federation Service Internet Information Services (IIS) Secure Sockets Layer (SSL) server certificate chains to a root that is trusted by the Federation Service Proxy.

Ensure that the Federation Service Uniform Resource Locator (URL) that is configured in the Federation Service Proxy web.config uses the name that is the subject of the Federation Service IIS SSL server certificate.

Additional Data

Exception details:

System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> Attempted to perform an unauthorized operation.

   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

   at System.Web.Security.SingleSignOn.FederationServerSoapProxy.GetProxyTrustConfiguration(VersionInformation proxyVersion, VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& trustConfig)

   at System.Web.Security.SingleSignOn.LSPersistentState.GetPolicy(VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& data)

Debug logs:

2008-06-03T22:13:09 [VERBOSE] Processing HTTP GET: https://adfsaccount.adatum.com/adfs/ls/?wa=wsignin1.0\&wtrealm=urn:federation:treyresearch\&wct=2008-06-03T22:13:09Z\&wctx=https://adfsweb.treyresearch.net:8081/claimapp/\\https://adfsweb.treyresearch.net:8081/claimapp/default.aspx

2008-06-03T22:13:09 [VERBOSE] Received SignIn Request.

2008-06-03T22:13:09 [ERROR] Exception from GetProxyTrustConfiguration: System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> Attempted to perform an unauthorized operation.

   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

   at System.Web.Security.SingleSignOn.FederationServerSoapProxy.GetProxyTrustConfiguration(VersionInformation proxyVersion, VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& trustConfig)

   at System.Web.Security.SingleSignOn.LSPersistentState.GetPolicy(VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& data)

2008-06-03T22:13:09 [EVENTLOG] Error ExceptionFromFedServer (https://adfsaccount.adatum.com/adfs/fs/FederationServerService.asmx, GetProxyTrustConfiguration, ECF1FE79E51231DF48098E1044233FCBDABF04CC, System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> Attempted to perform an unauthorized operation.

   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)

   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)

   at System.Web.Security.SingleSignOn.FederationServerSoapProxy.GetProxyTrustConfiguration(VersionInformation proxyVersion, VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& trustConfig)

   at System.Web.Security.SingleSignOn.LSPersistentState.GetPolicy(VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& data))

 

As you can see, there is a problem with the client auth certificate somewhere.  I did a fair amount of double checking my steps - but everything looked correct and seemed to be checking out.  The doubt was starting to creep in - I started to wonder how much I knew about this stuff!  Then I remembered an issue that came up a few weeks ago. 

The diagnostic tool does check for the existence and proper permissions of the private key and will flag it - but it does so in the user context.  ADFS is operating under the machine context.  So when I look at the certificate or run some certutil commands against it - it all checks out because I'm in my user security context.  If I launch a CMD prompt with AT scheduler and run the same commands or run the Diagnostic tool - I find the error.  The local computer does not have permissions to the private key of the client authentication certificate.

I was able to re-issue the certificate and mark the private keys as exportable, then do an export/import operation from the user store to computer store and everything worked as expected.

Since Client Authentication certificates are commonly used for user operations vs. computer operations - it is easy to see how others could hit this very same problem.  Hopefully the errors and debug log entries will make this blog post discoverable for others hitting this.