AD FS 2012 R2 Web Application Proxy – Re-Establish Proxy Trust


In the Tailspintoys environment, the administrator (moi) was a bit slack.  They let the AD FS 2012 R2 proxy get into a bad state.  The AD FS Proxy was not contacting the AD FS server on the internal network, and this allowed the short lived authentication certificate to expire.  At this point the AD FS Proxy was “dead to me” as far as the AD FS server was concerned.  The internal AD FS server was OK, the issue was just with the proxy.

Bummer….

How do we fix this?  Actually before we dive into that, lets see what was going on first.  Please note that this post is specific for AD FS 2012 R2.  It does not aim to cover AD FS 2.0 or 2.1 at all.

Starting Point – What The AD FS Proxy Saw

On the WAP server, the Remote Access Management Console was not happy.   It was reporting error code 0x8007520C.

ADFS 2012 R2 ADFS Proxy - Error 0x8007520C

On the WAP server, EventID 422 was logged into the AD FS Admin log stating that it was unable to retrieve proxy configuration data from the Federation Service.  The details in the event stated: System.Net.WebException: The remote server returned an error: (401) Unauthorized.

ADFS 2012 R2 EventID 422 - ADFS Proxy Unable To Retrieve Its Configuration Data from The Federation Service

Starting Point – What AD FS Saw

On the AD FS server EventID 394 indicated what the AD FS server really though about the AD FS proxy:

The proxy trust certificate specified by thumpbrint {0} has expired.

ADFS 2012 R2 EventID 394 - The Federation Server Could Not Renew ITs Trust With The Federation Service

Since the federation server proxy could not renew its trust with the Federation Service, the recommended user action was:  To ensure that the federation server proxy is trusted by the Federation Service. If the trust does not exist or has been revoked, establish a trust between the proxy and the Federation Service using the Federation Service Proxy Configuration Wizard by logging on to the proxy computer.   This is detailed in EventID 276, which is again logged on the AD FS server.

ADFS 2012 R2 EventID 276 - Federation Proxy Was Not Able To Authenticate To the Federation Service

OK – so we need to re-establish the trust between the AD FS proxy and AD FS server.  How do we go about doing that?

Re-Establish AD FS Proxy Trust Using Remote Access Management Console

Interestingly enough there is no option presented initially in the GUI to re-configure the AD FS proxy.

WAP 2012 R2 Remote Access Management Console Does Not Show Configuration Wizard

Currently it knows that the wizard was previously executed, and this fact is stored in the registry.  As Georg discussed at MEC last year to allow the Remote Access GUI to re-run the wizard again, we need to edit the registry.  The registry value that we need to change is:

HKLM\Software\Microsoft\ADFS\ProxyConfigurationStatus

We need to set the ProxyConfigurationStatus REG_DWORD to a value of 1 (meaning “not configured”) instead of 2 (“configured”). Once that change is made, re-open the GUI.  No reboot is required.

The Remote Access Manager should now allow you to re-run the configuration wizard.

WAP 2012 R2 Remote Access Management Console Now Shows Configuration Wizard

For full details on this process, please see this post.  As a brief recap, we need to ensure that the federation service name is the AD FS endpoint that we defined when initially building the environment.

Re-Running WAP 2012 R2 Remote Access Configuration Wizard

Since there may have been certificate replacement/renewal since the initial server was deployed, be careful to select the correct certificate:

Re-Running WAP 2012 R2 Remote Access Configuration Wizard - Ensure Correct Certificate Is Selected

The summary screen indicates the PowerShell code that will be executed.  This is also captured later in the post for reference.

Re-Running WAP 2012 R2 Remote Access Configuration Wizard - Summary

And when complete, the trust is now re-established.

Now restart the AD FS service on the Proxy server, and check the AD FS event logs to ensure that they are all green ticky ticky!

Re-Running WAP 2012 R2 Remote Access Configuration Wizard - Completed

Alternatively, this can be done from PowerShell, this is shown below.

Re-Establish AD FS Proxy Trust Using PowerShell

EventID 276 shown above, notes that we can run the Install-WebApplicationProxy cmdlet to re-establish trust between the AD FS server and the WAP.  TechNet discusses this in the Install and Configure the Web Application Proxy Server section.  The certificate we want to use is already installed onto the server.  In my case it has the thumbprint of 3EFF626CD4CAECDB6F84DB5FB4FCF580ACF629E2 – note that yours *WILL* be different.

The command that was executed was:

Install-WebApplicationProxy –CertificateThumbprint 3EFF626CD4CAECDB6F84DB5FB4FCF580ACF629E2 -FederationServiceName adfs.tailspintoys.ca

Note that the username and password were not specified, since the cmdlet knows it needs an account that has permission on the AD FS server and it prompts for this required information.  This is the Tailspintoys\administrator account as shown below:

Using Install-WebApplicationProxy Cmdlet To Re-Establish Trust Wth 2012 R2 ADFS Server

After providing the credentials, the cmdlet does it’s thang:

Install-WebApplicationProxy Cmdlet Is Doing Its Magic

Finally finishing up with a Deployment Succeeded message.

Install-WebApplicationProxy Cmdlet -- Deployment Succeeded

 

Checking For Success

After the restarting the AD FS service on the proxy, success messages were then logged on both the AD FS server and the proxy.

On the AD FS proxy EventID 245 noted that the proxy was able to successfully retrieve its configuration:

ADFS Proxy 2012 R2 - Now Able To Successfully Retrieve Configuration From ADFS Server

And on the AD FS server EventID 396 was logged stating that the trust between the proxy and AD FS server was renewed.

ADFS 2012 R2 - The Trust between The Federation Server Proxy And The Federation Service Was renewed Successfully

Clients were now able to successfully authenticate through the AD FS proxy from the Internet.

Good job.  Time to go home for tea and medals!


Cheers,

Rhoderick

Comments (33)

  1. Andrew Shin says:

    I appreciate the article; this got my proxy back up and running. However, my question is…. Why would the auto-renewal fail? I’d hate to have to re-configure the WAP every two weeks to make sure the certificate is renewed. The only thing I noticed when
    I was running the Install-WebApplicationProxy command was that I had to add the account I was using for the command to the Domain Admins group for it to go through; otherwise, the command would fail with "unauthorized: Verify that the account has administrative
    rights on the Federation Server". This error occurred even when the account was a local admin on the internal AD FS servers. Only after adding the account to Domain Admins group did the command run successfully.

    Any advice would be appreciated. Having to run the command to reconfigure/renew the cert manually every 2 weeks or so is far from ideal.

  2. Hi Andrew,

    On the WAP server – do you get the application event log entries every minute saying that the WAP server successfully retrieved its configuration?

    Also, was the WAP server offline or running for the two weeks prior?

    Cheers,
    Rhoderick

  3. Ryan Powell says:

    Hi Rhoderick,

    We’re also seeing the same issue as Andrew – one of our WAP servers loses it’s trust with the ADFS farm every two weeks and we then have to manually run the Powershell commands to re-register it. We have looked in the event logs of both the WAP and ADFS servers
    and there is nothing else logged other than the 422 and 394 events indicating it cannot renew the trust. The certificate we’re using is valid and install in the personal store as we have read elsewhere is recommended.

    The WAP box sits in our DMZ but has port 443 allowed through the firewall for communication. Our second WAP box was configured identically but we have had no problems with this one at all.

    Do you have any other troubleshooting tips for identifying why the WAP becomes unregistered and how we can stop this from happening again?

  4. Hi folks,

    Can I ask if you have a load balancer as part of the ADFS solution, either in the DMZ or on the corporate network.

    If so what type is it, and what setup steps did you follow please?

    Cheers,
    Rhoderick

  5. DeWayne Gibson says:

    I have seen this happen with two ADFS environments that I set up, both with load balancers. The first was using an F5 BigIP appliance and the second is in Azure using ILB for the ADFS application servers. I have discovered that by not using load balancing
    this does not happen, but that is obviously not the solution I’m looking for.

  6. Agreed – that is not the behaviour that you want!

    On the LB that was balancing the ADFS servers, were you terminating SSL on the LB?

    Cheers,
    Rhoderick

  7. Craig Tolley says:

    I had the same issue, with a Kemp Loadmaster as the LB for the internal servers. The best practice guide from Kemp currently states to terminate and re-encrypt the SSL traffic. Turning off this setting allowed everything from the WAPs to continue working
    as it should.

    1. Baz says:

      Hi Craig, I’m in similar situation as your’s , Kemp LB is stopping communication between WAP and AD FS. Just taking of Re-encrypt worked for you or have you done anything more?.

      1. Baz – is that still in the latest version of the Kemp docs?

        Cheers,
        Rhoderick

  8. DeWayne Gibson says:

    >>On the LB that was balancing the ADFS servers, were you terminating SSL on the LB?

    I’m afraid I don’t understand this question. The load balancer I’m currently using is the ILB in Azure.

    Also wanted to note that this article says that a update rollup from last June fixed the problem but my AD FS servers have the rollup installed and its still happening.

    http://blogs.technet.com/b/applicationproxyblog/archive/2014/05/28/understanding-and-fixing-proxy-trust-ctl-issues-with-ad-fs-2012-r2-and-web-application-proxy.aspx#pi148362=1

  9. zascherl says:

    You are missing a step, that may or may not effect different users. Run this command on primary ADFS server before Installing the thumbprint on the proxy server: Set-AdfsSslCertificate -Thumbprint xxxxxxxxxxxxxxxxxxxxxxxxx I don’t know what happened in
    my environment but that is a nightmare to troubleshoot.

  10. Dan Schultz says:

    I expected problems after my lab VMs were offline for six months, and it was very cool to find the answer (easily) for the last problem. Rhod saves the day once again!

  11. Hi Zascherl – you need to be careful with that command. It should not be necessary in the situation that the WAP has lost its way as described above.

    Cheers,
    Rhoderick

  12. Thanks Dan !

    Glad to hear that you got fixed up with this!

    Cheers,
    Rhod

  13. Marius Ene says:

    Worked like a charm! Thanks a lot!

  14. DellGuy1 says:

    this worked in a double nlb environment, 2 proxy on dmz network and 2 internal adfs. however we are still unsure of the cause

  15. Billiam says:

    We’re seeing this issue every 2 weeks and only have a single ADFS Proxy and ADFS server, but for some reason the proxy trust certificate doesn’t auto-renew and we see no event logs suggesting an error in renewing or suggesting that it even attempted to
    renew. The first issue we see is that it failed to get the configuration from the ADFS server with the eventid 422

  16. Ally says:

    Experienced this issue after windows patching done. Never saw event 394, but did see the other events. The above method was how I fixed it.

  17. Amayacitta says:

    I experienced this issue with KEMP load balancers when doing re-encryption. By disabling SSL acceleration this now works fine. At this point SSL termination happens on the AFDS servers not the load balancer, load balancing is still taking place however,
    the KEMP just can’t see the payload.

    Root cause 6 in the below blog eluded to what the issue was.

    http://blogs.technet.com/b/applicationproxyblog/archive/2014/05/28/understanding-and-fixing-proxy-trust-ctl-issues-with-ad-fs-2012-r2-and-web-application-proxy.aspx

  18. Yes – that is one of the common causes for this.

    Cheers,
    Rhoderick

  19. Ranj Bassi says:

    Hi Rhoderick

    Please if you could help

    Currently setting up a Web Application proxy to publish our CRM externally. The WAP is non domain server in our DMZ and we have only allowed Port 80 and 443 inbound/outbound from the WAP to the internal ADFS 3.0 server which is a domain joined server and a
    member of our AD domain.

    Had to create a local DNS entry on our WAP server using the hosts file to our ADFS server (sts1.orgname.com) and was able to configure successfully the WAP role and publish applications.

    I get the event ID 245 to prove this is the case:

    "The federation server proxy successfully retrieved its configuration from the Federation Service ‘sts1.orgname.com’.

    However to publish CRM successfully externally some additional steps need to be completed regarding disabling URL translation and to perform this piece I need to open up powershell and run the Get-WebApplicationProxyApplication cmdlet. I run the same command
    as shown in this document

    Get-WebApplicationProxyApplication Name* | Format-List replacing Name* with our own organization published apps name.

    https://blogs.technet.microsoft.com/dynamicspts/2014/10/01/using-web-application-proxy-to-publish-dynamics-crm-2013-to-the-internet/

    However for some reason in Powershell it doesn’t recognize that command at all and I get the following error message:

    Get-WebApplicationProxyApplication : Web Application Proxy could not connect to the AD FS configuration storage and
    could not load the configuration. Make sure that the Web Application Proxy server can connect to the AD FS server, and
    if not, run the Install-WebApplicationProxy command.
    (0x80075213)

    Now when I configured the WAP role I created a local user on the internal ADFS server and put this user in the ‘administrators’ group of the server and used this account to perform the initial authentication when configuring the WAP server under the WAP configuration
    wizard when it asks to enter the credentials of a local administrator account on the federation server.

    Would this account be sufficient or would I need to create a domain account in our AD and add this user in the local administrators group on the ADFS server and then use this account to configure WAP?

    Any help on this would be most appreciated

  20. ANF Anamul says:

    It’s nice and supper.

  21. Nimesha Jayasooriya says:

    We are experiencing the same issue. Checked with the script all 4 is passed. However, issue occurs when primary wap server restart to apply windows updates yet secondary wap does not provide service at the same time? Any considerations? Our ADFS WAP servers are on Azure and connected with Azure ILB. Is there any configuration can male in ILB?

  22. DubaStep says:

    So is there a way to do this without terminating SSL at a Windows box, which is, well, a terrible idea for lack of anything better to say about it. There has to be. F5 seems to say there is: https://devcentral.f5.com/articles/big-ip-and-adfs-part-5-working-with-adfs-30-and-sni The mention of SSL tunneling vs. bridging wouldn’t be there otherwise. It doesn’t really seem to say a clear way to do this though. I’d rather not be terminating SSL connections from the outside world on a domain joined Windows machine, given the suspect way SSL is implemented on Windows in the first place where disabling different SSL ciphers and protocols is a regedit nightmare.

    1. DubaStep says:

      Actually, figured out my own answer. Terminate SSL in front of the proxy and then re-encrypt.

  23. J_Systems says:

    Seeing this as a real damn mess. We also reverse proxy using hardware load balancers, worked around the WAP issue where it couldnt install initially and setup the adfstrusted certificate by configuring another VIP allowing for pass-thru.

    Now these certificates are the royal damn pain.

    Where the hell is it documented how it connects to all servers in the farm to deploy the ADFS ProxyTrust certificates? I would have hoped it loops through all servers in the farm but there is a big inconsistency. If we need to script this on a bi monthly basis and monitor it, it sounds kind of loopy to me.

    Not everyone in the world gives a rats about NLB!

    1. I personally agree – the intent is not to force you to use Windows NLB!

      You can add comments to all of the documentation, there are feedback links along the bottom of all pages. For example:
      https://technet.microsoft.com/en-us/library/dn383647.aspx

      I’ll also follow up internally on this.

      Cheers,
      Rhoderick

  24. Taparshi says:

    For everyone who face the WAP loosing its trust with ADFS every now and then because of Certificate propagation issue – initialize DRS (Device Registration Service) …you just need to initialize it ,no need to configure… Please refer “Root Cause 4 – Proxy Trust certificate propagation issues across an AD FS 2012 R2 farm” from https://blogs.technet.microsoft.com/applicationproxyblog/2014/05/28/understanding-and-fixing-proxy-trust-ctl-issues-with-ad-fs-2012-r2-and-web-application-proxy/… I solved my issues with this…

    1. Taparshi – are your servers all fully updated? That was meant to have been fixed with Windows updates from June 2015.

      Cheers,
      Rhoderick

      1. Taparshi says:

        yep, That patch you mentioned was already there but we were still facing issues and that surprised me quite a bit… It went away as soon as we initialized DRS and never came back !

  25. Nico Thiemer says:

    Very useful article!

  26. Sujithkumar says:

    I am getting the following error message even after running the web application proxy wizard using Powershell

    “Install-WebApplicationProxy Unable to retrieve proxy configuration data from federation server. ”

    I followed all the links mentioned below but nothing worked out for me!! please help us in fixing this issue.

    1. Ensure that TCP 443 is open from the WAP to AD FS.

      You may want to hosts file to a single AD FS server to easy troubleshooting. That also removes potential LB issues.

      Cheers,
      Rhoderick

Skip to main content