Backend certificate issues – an unknown error occurred processing the certificate.

A common issue with UAG is having your users receiving an error when trying to launch an application from UAG (such as OWA, for example). The error is “An unknown error occurred while processing the certificate”:

image

It’s important to realize that this means that UAG had an issue when trying to validate the certificate presented by the backend server that it’s publishing (for example, the Exchange or SharePoint server). When UAG needs to publish a backend server that is using SSL, UAG is configured by default to inspect that certificate using the standard mechanisms (validity, trust-chain, integrity and CRL). The error indicates that this failed for some reason.

It could be that the certificate itself has been damaged in some way, but the most common cause for this happening is a problem with the certificates CRL/CDP. The CRL is typically published automatically on the CA server that created the certificate, and the Certificate has a CDP (Certificate Distribution Point) list pointing to one or more CRL locations:

image

This means that the UAG needs to have access to at least one of these locations, to be able to download the CRL file, and verify the certificate has not been revoked. In organizations that have a tight control over what UAG has access to (with a DMZ firewall, for example), UAG may not have access and fail the certificate check. In some instances, a routing or DNS change on the internal network may cause the CRLs to become unavailable, causing the same problem.

Another common cause for this is an incorrect setup of the server name on the UAG applications’ Web Server tab:

clip_image006

When UAG needs to fetch information or pages from the backend server, it will resolve the hostname specified in the Addresses tab above, and contact that server. If, however, the certificate that server has contains a hostname that is different than this, UAG will not agree to continue and display an error such as mentioned above.

The ideal solution to the CRL problem is to adjust or fix the routing/DNS/Network/Firewall in a way that will provide UAG full access to verify the certificate. For a name-mismatch situation, the best thing to do is adjust the hostname specified in Addresses to one that matches the name on the certificate. These steps may not always be possible, or may involve a complex procedure and change-management hurdles, so there is another workaround. UAG has a configuration that tells it whether to validate the Certificate, and whether to validate the CRL locations. To overcome this problem, simply set UAG to not check these. The procedure is as follows:

1. On the UAG Server, open the Registry Editor

2. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\WhaleCom\e-Gap\Von\URLFilter\Comm\SSL

3. To cancel the validation check in its entirety, modify the value of ValidateRwsCert from 1 to 0

4. To cancel just the CRL check, modify ValidateRwsCertCRL from 1 to 0

5. Close the Registry Editor

6. Activate the UAG configuration to make the change permanent (otherwise, a reboot will revert it)

7. If this is applied to a UAG array, the registry needs to be edited only on the array manager. The activation will push it to the other server/s

8. Restart the IIS service on the Forefront UAG server. If this is an array, this needs to be done on all members.

Naturally, this brings about the question whether this workaround is reasonable. Essentially, skipping the certificate check is less secure than letting UAG perform the check, so our official recommendation is to fix the cert access and not work around it. However, the real-life security risk is usually not very high. The purpose of this check is for UAG to verify that the server it is talking to is real, and a situation where it is not may occur only if an attacker points the UAG application towards a different internal server by using DNS poisoning, altering the HOSTS file on UAG or editing the UAG app. Naturally, to do this, the attacker will have to achieve a very high level of penetration, which would also allow him to change this registry key. In other words, the default setting will probably not protect your organization from any attacker who has been able to jeopardize the certificate trust.

If the problem described above still persists after applying the registry changes, then something else may be interrupting the SSL Handshake mechanism between UAG and the backend server. For example, the backend servers SCHANNEL mechanism may not be compatible with UAG for some reason. To investigate this, a UAG Server trace will be required, and possible an SCHANNEL trace as well. Here’s a list of various SSL related codes that might provide more info:

CERT_E_CHAINING

0x800B010A

CERT_E_CN_NO_MATCH

0x800B010F

CERT_E_EXPIRED

0x800B0101

CERT_E_PURPOSE

0x800B0106

CERT_E_REVOCATION_FAILURE

0x800B010E

CERT_E_REVOKED

0x800B010C

CERT_E_ROLE

0x800B0103

CERT_E_UNTRUSTEDROOT

0x800B0109

CERT_E_UNTRUSTEDTESTROOT

0x800B010D

CERT_E_VALIDITYPERIODNESTING

0x800B0102

CERT_E_WRONG_USAGE

0x800B0110

CRYPT_E_EXISTS

0x80092005

CRYPT_E_REVOCATION_OFFLINE

0x80092013

CRYPT_E_REVOKED

0x80092010

CRYPT_VERIFYCONTEXT

0xF0000000

NTE_EXISTS

0x8009000F

SEC_E_ILLEGAL_MESSAGE

0x80090326

SEC_E_INCOMPLETE_MESSAGE

0x80090318

SEC_E_INVALID_TOKEN

0x80090308

SEC_E_NO_CREDENTIALS

0x8009030E

SEC_I_CONTEXT_EXPIRED

0x90317

SEC_I_INCOMPLETE_CREDENTIALS

0x90320

SEC_I_RENEGOTIATE

0x90321

TRUST_E_BASIC_CONSTRAINTS

0x80096019

TRUST_E_CERT_SIGNATURE

0x80096004