Common Problems while Implementing HTTPS Inspection on Forefront TMG 2010 RC

1. Introduction

 

The HTTPS Inspection feature on TMG 2010 can protect internal client workstation from accessing non legitimate HTTPS web sites. The whole idea is to avoid that client open a SSL tunnel with the destination server and the content that pass through this tunnel not being inspected, causing a potential way for malicious code to pass through and reach the client workstation.

The goal of this post is to explain the most common scenarios where client workstation might have issues when HTTPS Inspection is implemented on TMG 2010.

Note: For the implementation steps use the TMG Documentation page at Microsoft TechNet for more details.

2. Understanding the Inspection Process

 

In order to address the most common problems while implementing HTTPS inspection you need to understand how the HTTPS inspection works. The basic flow process happens as shown below:

Figure 1 – Basic HTTPS Inspection Flow

1) TMG reads configuration from storage via the configuration manager. This includes reading HTTPS inspection CA certificate. Configuration manager is a class implemented in msphlpr.dll module – this module is loaded in Firewall service memory space.

2) Client initiates connection (HTTP CONNECT request for full proxy, TCP server:443 for transparent client).

3) TMG makes SSL handshake with the destination server.

4) TMG checks server certificate according to HTTPS inspection configuration (certificate policy and exclusion checks) and works as follows:

a. Site should be blocked – send error page to client (full proxy clients will show it)

b. Site belongs to the exclusion list – close the connection with server and opens a new one, sends “200 Connected” to client (for full proxy clients). Client and server make SSL handshake and continue regular SSL.

c. Site is inspected and not blocked – make SSL handshake with client.

5) TMG makes SSL handshake with the client workstation.

The options that govern the inspection process can change according to the settings that you choose for your HTTPS inspection. For example: the certification validation process can perform the tests specified in the figure below, however it is up to the administrator which options he wants to select or not:

Figure 2 – Certification validation policy

There are some other checks that are not exposed in the UI, such as:

· Name mismatch

· Server certificate is not trusted

· Server certificate type (if it is not for server authentication for example)

3. Common Problems

Any security measure that is implemented in an enterprise can cause side effects that can raise questions from the end users about why something that was possible in the past is not working anymore. End users may assume that because the browser uses HTTPS to connect to the site, the connection is secure, which it is not a fully true statement since the web site can be using a bogus certificate or contain malicious code.

While the HTTPS inspection can protect internal clients from accessing suspicious sites that uses HTTPS, it can also cause end users to complain that they are unable to access some HTTPS web sites. This can indeed happen in some scenarios, such as the ones that will be explained below:

Question 1) I’m trying to browse a HTTPS web site and I’m receiving an error saying: The page cannot be displayed - Error Code 502 (Proxy Error) – the certification authority that issued the SSL Server certificate supplied by a destination server is not trusted by the local computer.

Cause: This problem happens when TMG does not trust the certificate of the SSL web site. For the most web sites that use commercial certificate authority this error shouldn’t happen. If the user really needs to access this web site, TMG administrator will need to import the CA certificate into the TMG local trusted root certificate store. Another workaround would be to add the site to the inspection exclusion list, and set certificate validation to “No validation”.

Question 2) I’m trying to browse a HTTPS web site and I’m receiving an error saying: The page cannot be displayed - Error Code 502 (Proxy Error) – the name on the SSL server certificate supplied by a destination server does not match the name of the host requested.

Cause: There are special cases where the “name mismatch” error that can happen and the conditions are:

· Web server uses a wild card certificate (*.domain.com for instance)

· Client is a transparent client (so accessing the web server using its IP address)

· Client is web proxy, but accesses the web server using its IP address

· Reverse name resolution (IP to name) of the web server fails from TMG

In that case, TMG needs to perform a reverse name resolution to identify any name mismatch.

However if the reverse name resolution fails for some reasons, TMG can’t complete the name mismatch validation. If this is a valid site and the end user needs to access the recommendation is to add the *.domain.com to the list of destinations exempted from inspection, and set the certificate validation to “No validation”.

Question 3) All HTTPS web sites that I’m trying to access the following message appears on IE:

Figure 3 – Web site security warning.

Cause: The most common cause for this error while accessing all HTTPS web sites is because the client workstation doesn’t trust the certificate that TMG is using. The CA certificate (e.g. self signed certificate) used by TMG must be deployed on the client, otherwise the client won’t trust the certificate issued by TMG on behalf of the web server. Read Deploying the HTTPS inspection trusted root CA certificate to client computers from TMG Documentation on TechNet for more information on how to deploy the CA certificate to the clients.

Question 4) All HTTPS web sites that I’m trying to access from Firefox I receive the error below (it works if I access using Internet Explorer):

Figure 4 – Error while browsing HTTPs sites through a third party browser.

Cause: Some third party browsers such as Firefox have their own trusted root certificate store and does not use the local windows trusted root certificate store while browsing Internet. Consult the third party browser documentation to see how to install the root certificate that TMG is using for HTTPS inspection. The reason why it works with Internet Explorer is because Internet Explorer consults the local Windows trusted root certificate store, therefore if the root CA certificate used by TMG is already deployed using group policy, then IE will trust the CA.

Question 5) My client workstations are using TMG Client because I want them to receive the balloon notification below while HTTPS inspection is happening:

Figure 5 – Message from TMG Client.

Users are saying that this notification just appears once for the same web site. Is that correct?

Cause: By default the notifications are cached for 12 hours. This means that a notification for a same site won’t reappear before 12 hours (if client machine is not rebooted during these 12 hours). The value can be changed using in the registry:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RAT\Stingray\Debug\FwcMgmt]

"FWC_MGMT_HTTPS_TEMPORARY_DISABLED_TIMEOUT"=dword:2932E00

The current value is 2932E00 which equals 43200000 = 12 hours in milliseconds. This setting is used by the TMG Client UI (which is responsible for showing the notification), no need to restart the process – this will take effect on the next notification arrival.

Question 6) The site that I'm trying to access requires client certificate authentication, but when I try to browse I got the following error message:

Figure 6 – Error while browsing a site that requires client authentication.

TMG live logging shows the following message:

Cause: TMG doesn’t support this scenario because it doesn’t own the client certificate. The workaround is to add site to exclusion list with any mark (“Validation” mark is recommended).

4. Conclusion

In this post we discussed the basic functionality of HTTPS Inspection on TMG 2010 and the most common deployment problems that you might face while deploying this feature.

Authors

Yuri Diogenes

Sr Security Support Escalation Engineer

Microsoft CSS Forefront Edge Team

Bala Natarajan

Sr Security Support Escalation Engineer

Microsoft CSS Forefront TMG Beta Team

Technical Reviewer

Eric Detoc

Escalation Engineer

Microsoft CSS Forefront TMG Beta Team