Troubleshooting Tips for VPN Client Access on ISA Server 2006

1. Introduction

I wrote some time back about an issue on ISA Server related to VPN and on that post I emphasized the following:

“…ISA Server VPN component uses Windows RRAS component and that when the ISA is isolated from the main problem, the troubleshooting line needs to follow RRAS tools and techniques.”

This is what boils down to on the most case that I work where ISA Server is being used as VPN Server. Troubleshooting on the ISA Server side most of the time is minimum, most of the time on the ISA Server side is just a matter of configuration, or should I say misconfiguration. The big and deep troubleshooting involve other elements around:

· RRAS

· Networking in general

· Connectivity with the DC (or RADIUS Server)

This week I got a couple of VPN cases that were really interesting and I would like to share with you the troubleshooting steps and solution for those.

2. Scenario 1 – VPN Client was unable to authenticate

In this scenario, all VPN settings were correct on ISA and on client; on ISA Server logging we just see error 0x80074E20, which means “A connection was gracefully closed in an orderly shutdown process with a three-way FIN-initiated handshake” (see error codes for more information). Basic connectivity with the DC was also good…but, basic means only a ping with a reply. No other device in between internal NIC of ISA and the DC, full communication was allowed.

Investigating further it was possible to see the following:

· Event Viewer

Event Type: Error

Event Source: Microsoft Firewall

Event Category: None

Event ID: 21137

Date: 9/24/2009

Time: 5:51:18 PM

User: N/A

Computer: ISACONTN1

Description:

The connectivity verifier "Domain Controller" reported an error when trying to connect to DCCONT.contoso.msft.

Reason: The request has timed out.

Event Type: Error

Event Source: NETLOGON

Event Category: None

Event ID: 5719

Date: 9/24/2009

Time: 5:54:27 PM

User: N/A

Computer: ISACONTN1

Description:

This computer was not able to set up a secure session with a domain controller in domain CONTOSO due to the following:

There are currently no logon servers available to service the logon request.

This may lead to authentication problems. Make sure that this computer is connected to the network. If the problem persists, please contact your domain administrator.

ADDITIONAL INFO

If this computer is a domain controller for the specified domain, it sets up the secure session to the primary domain controller emulator in the specified domain. Otherwise, this computer sets up the secure session to any domain controller in the specified domain.

· RRAS Tracing (need to run “netsh ras set tracing * enable” in order to have those files under %systemroot%\tracing)

Content of the file IASSAM.LOG:

[924] 09-24 17:53:39:429: NT-SAM Names handler received request with user identity CONTOSO\administrator.

[924] 09-24 17:53:39:429: Username is already an NT4 account name.

[924] 09-24 17:53:39:429: SAM-Account-Name is "CONTOSO\administrator".

[924] 09-24 17:53:39:429: NT-SAM Authentication handler received request for CONTOSO\administrator.

[924] 09-24 17:53:39:429: Processing MS-CHAP v2 authentication.

[924] 09-24 17:54:27:198: LogonUser succeeded.

[924] 09-24 17:54:27:198: NT-SAM User Authorization handler received request for CONTOSO\administrator.

[924] 09-24 17:54:27:198: Using native-mode dial-in parameters.

[924] 09-24 17:54:27:198: Sending LDAP search to DCCONT.contoso.msft.

[924] 09-24 17:54:27:198: LDAP ERROR in ldap_search_ext_sW. Code = 81

[924] 09-24 17:54:27:218: Extended error string: (null)

[924] 09-24 17:54:27:218: Retrying LDAP search.

[924] 09-24 17:54:27:618: Failed to connect to the cached DC, try DC locator ...

[828] 09-24 17:54:31:284: NT-SAM Names handler received request with user identity CONTOSO\administrator.

[828] 09-24 17:54:31:284: Username is already an NT4 account name.

[828] 09-24 17:54:31:284: SAM-Account-Name is "CONTOSO\administrator".

[828] 09-24 17:54:31:284: NT-SAM Authentication handler received request for CONTOSO\administrator.

[828] 09-24 17:54:31:284: Processing MS-CHAP v2 authentication.

[828] 09-24 17:54:31:284: LogonUser succeeded.

[828] 09-24 17:54:31:284: NT-SAM User Authorization handler received request for CONTOSO\administrator.

[924] 09-24 17:54:50:922: Failed to connect to the DC discovered by DC locator, try DC enumerator ...

[828] 09-24 17:55:02:979: Using native-mode dial-in parameters.

[828] 09-24 17:55:02:979: Could not open an LDAP connection to domain CONTOSO.

[924] 09-24 17:55:02:979: Could not open an LDAP connection to domain CONTOSO.

[828] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

[828] 09-24 17:55:03:089: Retrying LDAP search.

[828] 09-24 17:55:03:089: Could not open an LDAP connection to domain CONTOSO.

[828] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

[924] 09-24 17:55:03:089: NTDomain::getConnection failed: This operation returned because the timeout period expired.

By this time you can be thinking: how we have basic connectivity and it doesn’t reach the DC? That was the key question. The next step was a simple test: try to change the groups that had permission to access VPN and user a local group, from ISA itself. In order to do that I created a local group called VPNUsers, added the local administrator on it (for testing purpose) and try to add the CONTOSO\Domain Users when….got the following error: The RPC server is unavailable. Ok, now it makes sense and to start troubleshooting this I used:

Troubleshooting RPC Endpoint Mapper errors using the Windows Server 2003 Support Tools from the product CD

It turns out that the issue was related to the binding order of the network adapter. On ISA Server we had the following binding order:

 

 

Figure 1

It is strongly recommended that the internal network (in this case LocalCorp) is on the top. This can avoid many issues such as this one. Here are some references on that:

· Delay in NetBIOS connections from a multi-homed computer

· How to properly configure network binding order on a BackOffice Small Business Server system

· DNS: Valid network interfaces should precede invalid interfaces in the binding order

After changing the binding order it worked like a charm!

3. Scenario 2 – Unable to Authenticate using RSA VPN Client

First and foremost, even before start touching on ISA Server, it is very important to review the article below from RSA about how to configure ISA Server 2006 to use RSA for VPN authentication purpose:

https://www.rsa.com/rsasecured/guides/imp_pdfs/Microsoft_ISA2006_AM7.1_VPN.pdf

By following the above article you can pretty much configure all you need in order to make the VPN work with RSA. However in this case, although everything was done, it was not authenticating. The key here it was that in this case the authentication was done using RADIUS.

By looking the Netmon trace while authentication was happening (internal NIC of ISA) we could see that ISA (10.20.20.1) keep sending requests to the RADIUS Server (10.20.20.20) without answer:

Figure 2

According to the system administrator this RADIUS is being used for other purposes and it only fails with ISA. Going to the RADIUS Server it was possible to see the following event on Event Viewer:

Figure 3

That was simple, right? What happens is that if the RADIUS Server doesn’t have the RADIUS Client configured in there, it will fail to answer the request. To fix that we just need to add ISA Server as RADIUS Client and make sure that the secret between them is the same. Here are some references:

Event ID 13 — RADIUS Client Configuration

Error 930; The authentication server did not respond to authentication requests in a timely fashion

4. Conclusion

These are more examples about how troubleshoot VPN on ISA Server 2006. Try to not focus so much on ISA itself because many times ISA is not the culprit.