ISA Server 2006 SP1 shipped last week and it represents a big milestone for the product. The troubleshooting tools that are now built in will help the administrators effectively test their publishing rules prior to deploy published server to the public. But, what if everything looks good in the “testing environment” and when deploys to the external public it is failing?
An ISA administrator was saying that when he used the ISA Server 2006 SP1 Test Button, it shows that the publishing rule was configured correctly; much like Figure 1 below.
Figure 1 – Web Publishing Test results.
The problem is that when a user connected, he was receiving error (operation timed out) while accessing the web site from the Internet.
What Should I Expect from Test Button?
The first thing we need to do is set proper expectations about the Test Button feature. Here are some key aspects of this option:
What does it do?
1. Validates that TCP/IP connection can be established
- Hostname is resolvable
- Host is accessible on ports defined in the rule Bridging tab
2. Verifies server certificate information
- Cert not expired, not revoked
- Cert issued by a trusted CA
- Cert issued to the right name (with SAN support)
3. Validates that an HTTP GET request can pass through:
- ISA auth-delegation compatible with server auth requirements
- Published paths exist (tested only of auth not required)
- Published server accepts names forwarded in host header (as configured in the rule To tab)
What are the known limitations?
1. Internal rule conflicts are not checked
- It doesn’t test authentication; it only tests that IIS / ISA authentication methods are compatible
Web Publishing Scenario
The scenario was a basic HTTP web publishing rule with no SSL involved. The Web Server was using IIS7 and was working just fine for internal users.
Figure 2 – Web Publishing Scenario
Note: the dotted line represents the uplink between the switches.
The first troubleshooting step we performed was to understand why the client was getting time out from outside. To do that we enabled Live Logging and found the following error:
Figure 3 – Live logging error while accessing the IIS Server.
The 10060 is clear by definition: ISA couldn’t get an answer from the published server. But this was really trick since we could access the web server from the ISA Server itself. To really understand what was happening it was necessary to attach netmon to the internal nic of the ISA Server.
Analyzing the Netmon Result
After starting the netmon trace and reproducing the issue from the outside client, the following behavior was noticed:
1. ISA Server sends the TCP SYN to establish the handshake on port 80:
2. ISA Server does no receive an answer and try it again:
3. …and again:
5. After all those attempts it times out:
It seems clear from the network capture that the IIS Server is not answering the request. Oddly enough, the IIS admin responded that the IIs server was answering the request, and provided this capture from the IIS server:
1. IIS Server receives the TCP SYN on port 80:
2. It answers the SYN with the ACK:
3. It receives another SYN:
4. It answers again”
5. ..and this happens until IIS resets the connection since the handshake was not established successfully:
The question that arises now is: if ISA Server is sending packets to the IIS server and the IIS server is replying, then where is the traffic failing between ISA and IIS? Before we think too deeply on that question, we’ll need to step back and understand the topology. The external client is identified in this lab as 192.168.0.107 and the reason why the IIS Server is receiving this address as the source IP in the packet is because of the following option that it is selected in the publishing rule:
Figure 4 – ISA Server Publishing option.
According to the customer, this option was selected because they want to audit IIS Logs and know what external IPs is accessing the web site. A couple of tests were performed at the IIS server to better understand how the IIS sends traffic to non-local subnets:
It timed out from the beginning, so it was time to check the routing table:
The answer for this problem it is showed above in red, the default gateway. Now for sure we know what was happening. Check the traffic flow in the diagram below:
Figure 5 – Traffic flow.
As you can see above, client sends the HTTP GET, the ISA Server accepts the request since it matches with the web publishing rule, it tries to establish the TCP handshake on port 80 with the IIS Server sending the TCP SYN, the IIS Server look to the source IP and notice that it is an external IP and sends the TCP SYN ACK to the default gateway which is the router.
The conclusion here is that the switch layer 3 that the IIS Server was using as default gateway didn’t have the correct routing table configured to allow the external IP go back through the ISA Server. The possible solutions were:
Change the IIS Server to use ISA Server as default gateway.
Add specific router in the layer 3 switch for the internal networks and change the layer 3 switch to use ISA Server as default gateway.
Change the ISA Server publishing rule to allow the requests appears that are coming from the ISA Server.
The first option was the one choose by the customer and it fixed the issue. The lesson learned here are:
The Test Button is a great resource to validate the configuration. But since the traffic is NOT 100% the same as when the client access from outside, the test can’t check everything.
A good understanding of your own network topology could avoid issues like that to happen.
Security Support Engineer – Microsoft CSS Forefront (ISA/TMG) Team
Microsoft Forefront (ISA/TMG) Sustained Engineering Team
Microsoft Forefront (ISA/TMG) Sustained Engineering Team