Today, more and more businesses rely on their Internet Service Providers (ISP) to handle their outside Internet communications. Sending emails, browsing the web and any other web related actions are essential business infrastructure services that are only available as long as the ISP line is up and running.
Keeping a stable, available and reliable outside Internet connection is one of the critical tasks on every administrator’s check list.
Forefront TMG provides a new capability called ISP redundancy which enables utilizing not one, but two ISP links for external connectivity, either for traffic load balancing or as a failover backup.
This post explains an important aspect in the ISP Redundancy configuration: “Persistent Routing Rules”, which is required for smooth operation of the ISP redundancy feature, and explains the way TMG decides which connection will use which ISP.
Load Balance mode, algorithm description
When selecting the Load Balance mode in the ISP Redundancy Wizard (as seen in the screenshot), it is not obvious which connection will go through which ISP (this is handled automatically by TMG) but in case you are curious…
We calculate a hash value based on the source IP and the destination IP, resulting in a number between 0 and 100. In the case that the result is below the percentage defined for ISP link 1, TMG will use link 1 for this connection, otherwise, ISP link 2 will be used.
TMG performs the calculation when establishing every outgoing connection.
This form of calculation assures session stickiness – all connections for a specific (Source, Destination) pair will go through one link.
Once you complete the ISP Redundancy wizard located in Networking -> ISP Redundancy:
The next step left to complete the configuration of the ISP Redundancy feature: both NICs should be configured properly.
A default gateway must be defined on the NICs connected to both ISPs. Otherwise, when the ISP that is configured with the only default gateway is down, there is no route to the Internet.
Windows alerts the user with the warning below when defining more than one default gateway on the machine. In our case it’s OK.
Note: Traffic originating from the local-host is not affected by the ISP Redundancy feature. This includes DNS requests from the local-host, initiated by the proxy.
Due to the fact that the OS selects the DNS servers to use with no reference to the NIC they are configured on, there might be a scenario that a query to the DNS server of ISP-2 will be sent through ISP-1.
A common behavior of ISPs is not to answer DNS requests that are not from their network as shown in the drawing below.
The solution to the scenario above is to complete the configuration of ISP Redundancy by adding a persistent static route for each DNS IP address configured on the external network adapters on every Forefront TMG server.
This is required to ensure that DNS requests are routed through the proper network adapter.
Adding the persistent static route:
C:\> ROUTE [-f] [-p] [-4|-6] command [destination] [MASK netmask] [gateway] [METRIC metric] [IF interface]
C:\> route -p add 192.168.5.1 mask 255.255.255.0 192.168.1.1 metric 1
For more options like flushing the IP Routing table or to delete/modify an IP Routing table entry, use the route command with no arguments. This displays the various options for the route command.
The last step in configuring Forefront TMG for ISP redundancy involves turning off the automatic metrics option. Instead, you must define a different static metric for each network adapter.
If automatic metrics is not turned off, when the operating system recalculates the network selection, it may cause misalignment with Forefront TMG route cache functionality. This can interrupt communication, such as UDP communications used typically by Instant Messenger network discovery phase.
To turn off the Automatic Metric feature:
- In Control Panel, double-click Network Connections.
- Right-click a network interface, and then click Properties.
- Click Internet Protocol (TCP/IP), and then click Properties.
- On the General tab, click Advanced.
- To specify a metric, on the IP Settings tab, clear the Automatic metric check box, and then enter the metric that you want in the Interface Metric field. It is recommended to define a lower interface metric value for the network adapter set to handle more traffic in ISP redundancy load balancing mode, or set as the primary link in failover mode.
For more information regarding Automatic Metric - http://support.microsoft.com/kb/299540/
Important To Remember
1. ISP Redundancy is only functional for a NAT relationship: testing connectivity from the local-host will not work and an admin may fail to understand why.
2. Because of the specifics of the load balancing algorithm explained above, it is possible that a bandwidth-consuming session will be assigned to the “slower” ISP connection and will lead to an incorrect load balancing ratio.
3. It is highly recommend leaving the “Connectivity detection” field in ISP settings as enabled. This value should be changed for troubleshooting purposes or in special cases only. Changing it will cause a malfunction in the failover mechanism.
Question: Where can the administrator see the ISP Redundancy behavior?
Answer: The information is presented in TMG Dashboard à Network Status. :
Question: In what cases can I use the ISP Redundancy feature?
Answer: ISPR can be used for any internet traffic, not only HTTP. However, the web application filter is only used for HTTP / HTTPS traffic.
Question: Can I use ISP Redundancy in a single NIC configuration?
Answer: Yes, to configure ISPR with a single NIC you should choose the same NIC for both ISPs, but specify separate subnets for each of them. This is true for Load Balancing mode and for Failover mode.
Author: Alon Yardeni, Program Manager, Microsoft Forefront TMG.
Reviewers: Evgeny Katz, Gabriel Koren, Meir Feinberg, Nathan Bigman