Troubleshooting Client Access to an IAG Portal

1. Introduction

 

One common problem that is happening these days when customers start to deploy IAG Portal is due one simple fact called “certificate”. There are many mistakes that can happen while implementing a new certificate to assign to an IAG Portal. This post will explain the symptom, how to identify the problem and how to fix it.

 

2. Symptom

 

The scenario was: customer has the main IAG Portal working and he needs to create a new webmail portal for the company. This webmail portal will be accessed from outside by using the name https://webmail.contoso.com. When outside users try to access this web site nothing comes up, the Internet Explorer returns the error message saying: Page Cannot Be Displayed. Here the repro lab for this scenario:

 

Figure 1 – Scenario used to reproduce the issue.

 

3. Gathering More Information on the Client Side

 

When dealing with IAG problems like that, it is important to validate first the following items:

· There is any edge firewall in front of the IAG?

· If there is one, is it allowing the traffic to the IAG box?

 

If it is allowing the traffic, then check if the client is able to establish the SSL Handshake with the IAG Server. To do that you can use the following approach:

 

· Install Fiddler and Netmon in the external workstation that you are using to make the connection.

· Launch Fiddler and Netmon in this workstation.

· On Netmon, start a new capture.

· Open Internet Explorer and try to access the URL.

 

Here the result for this example:

 

1. On Netmon you will see a pattern like that:

 

TCP Three Way Handshake happening:

192.168.0.50 192.168.0.71 TCP TCP: Flags=.S......, SrcPort=3190, DstPort=HTTPS(443), Len=0, Seq=3946337071, Ack=0, Win=65535 (scale factor not found)

192.168.0.71 192.168.0.50 TCP TCP: Flags=.S..A..., SrcPort=HTTPS(443), DstPort=3190, Len=0, Seq=2067442050, Ack=3946337072, Win=16384 (scale factor not found)

192.168.0.50 192.168.0.71 TCP TCP: Flags=....A..., SrcPort=3190, DstPort=HTTPS(443), Len=0, Seq=3946337072, Ack=2067442051, Win=65535 (scale factor not found)

 

Client tries to initiate the SSL Handshake:

192.168.0.50 192.168.0.71 SSL SSL

 

Destination Server Finishes the Connection

192.168.0.71 192.168.0.50 TCP TCP: Flags=F...A..., SrcPort=HTTPS(443), DstPort=3190, Len=0, Seq=2067442051, Ack=3946337142, Win=65465 (scale factor not found)

 

2. Using the Fiddler result, you can also look to the following to confirm the behavior:

 

 

Figure 2 – Fiddler result.

 

As you can see above, the client sends the SSL ClientHello to initialize the handshake many times (see left panel). On the right panel (Session Inspector) you will see the details about the ClientHello handshake and the ciphers that are supported by the client. We are expecting the ServerHello that never comes back.

 

4. Reviewing the IAG Configuration

 

At this point we know that something is not correctly configured in the IAG Trunk and we also know that it is related to the SSL since we were unable to see the ServerHello from the IAG. Therefore the first thing that needs to be checked is the Certificate.

 

Open the IAG Configuration Console, click in the Webmail trunk and click in Configure under Advanced Trunk Configuration. In the General tab you will see the window below:

 

 

Figure 3 - Certificate for the Webmail Trunk.

 

Unfortunately this doesn’t means that the certificate is good since the IAG doesn’t do certificate validation (like ISA Server does). But, here you will see at least which certificate you are using on this trunk. After look at this the next step is to open the IIS Manager, right click in the webmail virtual directory choose Properties, Directory Security and then click View Certificate. Here what I have on this case:

 

 

Figure 4 – Certificate without the private key.

 

Here it is the problem, which is a very common mistake. What happened here is that the user just imported the CER file to the personal store, which doesn’t have the private key. The IAG doesn’t complain about that (like ISA Server does), it accepts and you might think that everything is good at that point.

 

5. How to Fix It

 

The first step to fix is to remove that certificate from the IIS Virtual Directory by following the steps below:

 

1. Open the IIS Manager.

2. Right click in the virtual directory that has the wrong certificate (in this case webmail), choose Properties.

3. Click in Directory Security.

4. Click in Server Certificate.

5. Click in Next in the first window and choose the option Remove the Current Certificate and click Next.

6. Click again Next until you have the option Finish.

 

Also, make sure that the certificate is deleted from the local store. To do that, follow the steps below:

 

1. Click Start, Run and type MMC.

2. Click in File, Add Remove Snap In and click in Add.

3. Choose Certificates and click Add.

4. Choose Computer Account, click Next (leave the default option) and click Finish.

5. Click Close and click OK.

6. Expand Certificates, Personal and Certificates.

7. Right click in the certificate that you excluded from IIS and choose Delete.

8. Confirm saying Yes.

 

Now you need to obtain the certificate that has the private key, the file will have the PFX extension. If you are using Windows Certificate Services, make sure that the following option is enabled when you are exporting the certificate:

 

 

Figure 5 – Exporting the certificate with the private key.

 

After export the certificate with the private key, import it again in the IAG box. After import the certificate on the Personal Store of the IAG Server, open the IAG Configuration, make sure that under the advanced configuration trunk you have the certificate there (should be same as figure 3) and activate the configuration. This activation will allow the IIS commit the changes and bind the new certificate to the virtual directory. Now, if you open the IIS Manager, the certificate will appear like this:

 

 

Figure 6 – Correct Certificate.

 

6. Testing the Access to the Portal

 

If you use the same tools described in the session 3 while opening the web site you will see the following result:

 

1. On Netmon you will see a pattern like that:

 

TCP Three Way Handshake happening:

192.168.0.50 192.168.0.71 TCP TCP: Flags=.S......, SrcPort=3241, DstPort=HTTPS(443), Len=0, Seq=3798431583, Ack=0, Win=65535 (scale factor not found)

192.168.0.71 192.168.0.50 TCP TCP: Flags=.S..A..., SrcPort=HTTPS(443), DstPort=3241, Len=0, Seq=2984497643, Ack=3798431584, Win=16384 (scale factor not found)

192.168.0.50 192.168.0.71 TCP TCP: Flags=....A..., SrcPort=3241, DstPort=HTTPS(443), Len=0, Seq=3798431584, Ack=2984497644, Win=65535 (scale factor not found)

 

SSL Handshake:

192.168.0.50 192.168.0.71 SSL SSL

192.168.0.71 192.168.0.50 SSL SSL

192.168.0.50 192.168.0.71 SSL SSL

192.168.0.71 192.168.0.50 SSL SSL

 

2. Using the Fiddler the result may vary because of the certificate, but the core part is that we have an answer from the server now:

 

 

Figure 7 – Now we can see the answer from the IAG Server.

 

Above you have an error represented by the red circle because my external workstation doesn’t trust the certificate authority that issues the webmail.contoso.com certificate. If you look to the above figure, you will see that the right panel shows the attempt to get the certificate for dallas.cotnoso.com, which is the CA that issued this certificate.

 

When the workstation doesn’t trust the CA that issued the certificate a pop up window like the one below appears warning about that:

 

 

Figure 8 – Security alert.

 

7. Conclusion

 

The simple fact that you don’t import the correct certificate can cause this entire headache. This article covered the identification of the problem, the understanding why this happen and how to fix it. I hope you’ve enjoyed.