ISA Server 2006 SP1 – Problems that Go Beyond the Test Button
Introduction
ISA Server 2006 SP1 shipped last week and it represents a big milestone for the product. The troubleshooting tools that are now built in will help the administrators effectively test their publishing rules prior to deploy published server to the public. But, what if everything looks good in the “testing environment” and when deploys to the external public it is failing?
An ISA administrator was saying that when he used the ISA Server 2006 SP1 Test Button, it shows that the publishing rule was configured correctly; much like Figure 1 below.
Figure 1 – Web Publishing Test results.
The problem is that when a user connected, he was receiving error (operation timed out) while accessing the web site from the Internet.
What Should I Expect from Test Button?
The first thing we need to do is set proper expectations about the Test Button feature. Here are some key aspects of this option:
— What does it do?
1. Validates that TCP/IP connection can be established
- Hostname is resolvable
- Host is accessible on ports defined in the rule Bridging tab
2. Verifies server certificate information
- Cert not expired, not revoked
- Cert issued by a trusted CA
- Cert issued to the right name (with SAN support)
3. Validates that an HTTP GET request can pass through:
- ISA auth-delegation compatible with server auth requirements
- Published paths exist (tested only of auth not required)
- Published server accepts names forwarded in host header (as configured in the rule To tab)
— What are the known limitations?
1. Internal rule conflicts are not checked
- It doesn’t test authentication; it only tests that IIS / ISA authentication methods are compatible
Web Publishing Scenario
The scenario was a basic HTTP web publishing rule with no SSL involved. The Web Server was using IIS7 and was working just fine for internal users.
Figure 2 – Web Publishing Scenario
Note: the dotted line represents the uplink between the switches.
Gathering Data
The first troubleshooting step we performed was to understand why the client was getting time out from outside. To do that we enabled Live Logging and found the following error:
Figure 3 – Live logging error while accessing the IIS Server.
The 10060 is clear by definition: ISA couldn’t get an answer from the published server. But this was really trick since we could access the web server from the ISA Server itself. To really understand what was happening it was necessary to attach netmon to the internal nic of the ISA Server.
Note: you can also use ISA Data Packager in repro mode that already enables netmon and gathers additional information that can be used for troubleshooting purpose.
Analyzing the Netmon Result
After starting the netmon trace and reproducing the issue from the outside client, the following behavior was noticed:
1. ISA Server sends the TCP SYN to establish the handshake on port 80:
11:50:35.510 192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1192, DstPort=HTTP(80), Len=0, Seq=3373035932, Ack=0, Win=65535 (scale factor not found)
2. ISA Server does no receive an answer and try it again:
11:50:38.474 192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1192, DstPort=HTTP(80), Len=0, Seq=3373035932, Ack=0, Win=65535 (scale factor not found)
3. …and again:
11:50:44.483 192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1192, DstPort=HTTP(80), Len=0, Seq=3373035932, Ack=0, Win=65535 (scale factor not found)
4….and again:
11:50:56.520 192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1192, DstPort=HTTP(80), Len=0, Seq=266374158, Ack=0, Win=65535 (scale factor not found)
5. After all those attempts it times out:
11:51:26.493 192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1192, DstPort=HTTP(80), Len=0, Seq=327351070, Ack=0, Win=65535 (scale factor not found)
It seems clear from the network capture that the IIS Server is not answering the request. Oddly enough, the IIS admin responded that the IIs server was answering the request, and provided this capture from the IIS server:
1. IIS Server receives the TCP SYN on port 80:
192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1198, DstPort=HTTP(80), Len=0, Seq=333322946, Ack=0, Win=65535 (scale factor not found)
2. It answers the SYN with the ACK:
10.20.20.138 192.168.0.107 TCP TCP: Flags=.S..A..., SrcPort=HTTP(80), DstPort=1198, Len=0, Seq=1649575802, Ack=333322947, Win=8192 (scale factor not found)
3. It receives another SYN:
192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1198, DstPort=HTTP(80), Len=0, Seq=333322946, Ack=0, Win=65535 (scale factor not found)
4. It answers again”
29 10:00:07.573 10.20.20.138 192.168.0.107 TCP TCP: Flags=.S..A..., SrcPort=HTTP(80), DstPort=1198, Len=0, Seq=1649575802, Ack=333322947, Win=8192 (scale factor not found)
5. ..and this happens until IIS resets the connection since the handshake was not established successfully:
192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1198, DstPort=HTTP(80), Len=0, Seq=333322946, Ack=0, Win=65535 (scale factor not found)
10.20.20.138 192.168.0.107 TCP TCP: Flags=.S..A..., SrcPort=HTTP(80), DstPort=1198, Len=0, Seq=1649575802, Ack=333322947, Win=8192 (scale factor not found)
192.168.0.107 10.20.20.138 TCP TCP: Flags=.S......, SrcPort=1198, DstPort=HTTP(80), Len=0, Seq=3699107062, Ack=0, Win=65535 (scale factor not found)
10.20.20.138 192.168.0.107 TCP TCP: Flags=.S..A..., SrcPort=HTTP(80), DstPort=1198, Len=0, Seq=1649575802, Ack=333322947, Win=8192 (scale factor not found)
10.20.20.138 192.168.0.107 TCP TCP: Flags=..R....., SrcPort=HTTP(80), DstPort=1198, Len=0, Seq=1649575803, Ack=333322947, Win=0 (scale factor not found)
The question that arises now is: if ISA Server is sending packets to the IIS server and the IIS server is replying, then where is the traffic failing between ISA and IIS? Before we think too deeply on that question, we’ll need tostep back and understand the topology. The external client is identified in this lab as 192.168.0.107 and the reason why the IIS Server is receiving this address as the source IP in the packet is because of the following option that it is selected in the publishing rule:
Figure 4 – ISA Server Publishing option.
According to the customer, this option was selected because they want to audit IIS Logs and know what external IPs is accessing the web site. A couple of tests were performed at the IIS server to better understand how the IIS sends traffic to non-local subnets:
C:\Users\administrator.CONTOSO>tracert -d 192.168.0.107
Tracing route to 192.168.0.107 over a maximum of 30 hops
1 * * * Request timed out.
It timed out from the beginning, so it was time to check the routing table:
C:\Users\administrator.CONTOSO>route print
===========================================================================
Interface List
10 ...00 03 ff 06 fc 23 ...... Intel 21140-Based PCI Fast Ethernet Adapter (E
lated)
1 ........................... Software Loopback Interface 1
11 ...00 00 00 00 00 00 00 e0 isatap.{7B34369C-34DA-4775-9C7B-1CFEAF83257A}
12 ...02 00 54 55 4e 01 ...... Teredo Tunneling Pseudo-Interface
===========================================================================
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 10.20.20.5 10.20.20.138 276
10.20.20.0 255.255.255.0 On-link 10.20.20.138 276
10.20.20.138 255.255.255.255 On-link 10.20.20.138 276
10.20.20.255 255.255.255.255 On-link 10.20.20.138 276
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 10.20.20.138 276
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 10.20.20.138 276
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 10.20.20.5 Default
===========================================================================
The answer for this problem it is showed above in red, the default gateway. Now for sure we know what was happening. Check the traffic flow in the diagram below:
Figure 5 – Traffic flow.
As you can see above, client sends the HTTP GET, the ISA Server accepts the request since it matches with the web publishing rule, it tries to establish the TCP handshake on port 80 with the IIS Server sending the TCP SYN, the IIS Server look to the source IP and notice that it is an external IP and sends the TCP SYN ACK to the default gateway which is the router.
Conclusion
The conclusion here is that the switch layer 3 that the IIS Server was using as default gateway didn’t have the correct routing table configured to allow the external IP go back through the ISA Server. The possible solutions were:
— Change the IIS Server to use ISA Server as default gateway.
— Add specific router in the layer 3 switch for the internal networks and change the layer 3 switch to use ISA Server as default gateway.
— Change the ISA Server publishing rule to allow the requests appears that are coming from the ISA Server.
The first option was the one choose by the customer and it fixed the issue. The lesson learned here are:
— The Test Button is a great resource to validate the configuration. But since the traffic is NOT 100% the same as when the client access from outside, the test can’t check everything.
— A good understanding of your own network topology could avoid issues like that to happen.
Author
Yuri Diogenes
Security Support Engineer – Microsoft CSS Forefront (ISA/TMG) Team
Technical Reviewers
Jim Harrison
Microsoft Forefront (ISA/TMG) Sustained Engineering Team
Doron Juster
Microsoft Forefront (ISA/TMG) Sustained Engineering Team