Network port design and Exchange 2007 clusters.

  • Article

(An Exchange 2010 version of this article can be found here: https://blogs.technet.com/b/timmcmic/archive/2010/05/30/network-port-design-and-exchange-2010-database-availability-groups.aspx)

A question that I often see come up is how many network ports should I have in my clustered nodes and how should I use them.

I generally see three different hardware configurations:

  • Two network ports.
    • Usually 2 onboard or 1 onboard / 1 add-on.
  • Three network ports
    • Usually 2 onboard / 1 add-on.
  • Four network ports.
    • Usually 2 onboard / 2 add-on.

In some hardware there are now 4 port cards.  The information contained here can be expanded to include additional hardware / port configurations as they become available.

You’ll note that there is no configuration with a single network port – all clustered installations must at minimum have two network interfaces.  (Note:  VLANS to a single port are not two network interfaces).

When I’m advising customers the number of ports I recommend is dependant on the installation of Exchange. 

  • Exchange 2007 SP1 SCC (Single Copy Cluster).
    • Generally recommend the 2 or 3 interface installation.
  • Exchange 2007 SP1 CCR (Cluster Continuous Replication)
    • Generally recommend the 3 or 4 interface installation.

Using these recommendations I’ll break down their uses below using Windows 2008 clustering terminology.

Network Teaming

In the recommendations I’ll outline next you will see references to the use of network teaming.  It’s important to note that Microsoft does not support network teaming as this is hardware vendor supported and designed technology.  What it is though is a recognition that in absence of anyway to provide multiple client facing ports for Exchange network teaming does have a valid place in the overall high availability design.

When using network teaming, only the client facing network should be a teamed adapter and at all times the team created for NETWORK FAULT TOLERANCE.  Do not, for an Exchange instance, use any type of load balancing between ports.

For non-client facing networks it is not supported and not necessary to implement at network team (these would typically be your “heartbeat” networks).  (Refer to:  https://support.microsoft.com/kb/254101).  Windows clustering has the ability to balance and use all interfaces on the cluster designated for cluster use without the need to establish teaming for cluster / heartbeat communications.

From a support perspective any customer that establishes a teamed interface for the client side network should recognize that they may be asked to dissolve the team to support troubleshooting efforts.

Exchange 2007 SP1 SCC (Single Copy Cluster) – Two Network Ports

When using a single copy cluster with two network ports, your options are limited.  Consider the following design:

  • First port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
  • Second port set to “allow the cluster to use this network”.

 

Exchange 2007 SP1 SCC (Single Copy Cluster) – Three Network Ports

This option provides you with some additional flexibility and also allows you to mitigate issues on the client facing network.

  • First and second port configured in a fault tolerate team set for “allow the cluster to use this network” and “allow clients to connect through this network”.
  • Third port set for “allow the cluster to use this network”.

 

Exchange 2007 SP1 CCR (Cluster Continuous Replication) – Two Network Ports

When using a cluster continuous replication cluster with two network ports, your options are limited.  Consider the following design:

  • First port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
  • Second port set to “allow the cluster to use this network” and “allow clients to connect through this network”.

You’ll noticed that in this configuration both networks are set to “allow clients to connect through this network”.  This is necessary in order to establish the “private” network for use with log shipping functions.

To establish this network for log shipping functions, refer to the enable-continuousreplicationhostnames commandlet.  (https://technet.microsoft.com/en-us/library/bb690985.aspx / https://technet.microsoft.com/en-us/library/bb629629.aspx)

If used the replication service will first prefer to perform log shipping functions over the “private” network.  Should the private network be unavailable the replication service will resume log shipping functions over the “public” network.

 

Exchange 2007 SP1 CCR (Cluster Continuous Replication) – Three Network Ports

This option for CCR provides some additional flexibility.  We can use a minimum of three ports to assist us in mitigating two factors that affect the overall high availability of this solution.

  • More then one adapter assigned to the client facing network.
  • Establishing more then one path to replicate log files between the nodes.
  • Allow log replication to first prefer to “private” network dedicating the primary function of the “public” network to servicing client requests.

Considering using these ports in the following manner:

  • First and second port configured in a fault tolerant team set for “allow the cluster to use this network” and “allow clients to connect through this network”.
  • Third port set for “allow the cluster to use this network” and “allow clients to connect through this network”.

You’ll noticed that in this configuration both networks are set to “allow clients to connect through this network”.  This is necessary in order to establish the “private” network for use with log shipping functions.

To establish this network for log shipping functions, refer to the enable-continuousreplicationhostnames commandlet.  (https://technet.microsoft.com/en-us/library/bb690985.aspx / https://technet.microsoft.com/en-us/library/bb629629.aspx)

If used the replication service will first prefer to perform log shipping functions over the “private” network.  Should the private network be unavailable the replication service will resume log shipping functions over the “public” network.

 

Exchange 2007 SP1 CCR (Cluster Continuous Replication) – Four Network Ports

This option provides even greater flexibility in terms of minimizing single points of failure and assisting to ensure log shipping functions can be successful.

  • More then one adapter assigned to the client facing network.
  • Establishing more then one path to replicate log files between the nodes.
  • Allow log replication to first prefer to “private” network dedicating the primary function of the “public” network to servicing client requests.

I generally see the four port design implemented in two methods:

  • Method #1:
    • First port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
    • Second port set to “allow this cluster to use this network”.
    • Third port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
    • Forth port set to :allow the cluster to use this network” and “allow clients to connect through this network”.
    • Continuous replication host names used with ports three and four networks.
  • Method #2 (my personal preference):
    • First and second port configured in a network fault tolerant team set to “allow the cluster to use this network” and “allow clients to connect through this network”.
    • Third port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
    • Forth port set to “allow the cluster to use this network” and “allow clients to connect through this network”.
    • Continuous replication host names used with ports three and four networks.

Method 1 allows for individuals to not use network teaming on the public interface.  In this case the public interface is a single point of failure.  It does though allow for two secondary replication networks for log shipping functions providing three overall paths for replication service use.

Method 2, which is my personal preferred method, allows for individuals to establish a team on the public interface.  This provides high availability for the client facing network ports.  In addition, the remaining two interfaces are enabled for continuous replication host names.  This allows the replication service to have two secondary replication networks for log shipping functions providing three overall paths for replication service use.

You’ll noticed that in this configuration non-client facing networks are set to “allow clients to connect through this network”.  This is necessary in order to establish the “private” network for use with log shipping functions.

To establish this network for log shipping functions, refer to the enable-continuousreplicationhostnames commandlet.  (https://technet.microsoft.com/en-us/library/bb690985.aspx / https://technet.microsoft.com/en-us/library/bb629629.aspx)

If used the replication service will first prefer to perform log shipping functions over the “private” network.  Should the private network be unavailable the replication service will resume log shipping functions over the “public” network.

 

I hope this information was helpful as you consider the number of network ports for your Exchange clustered nodes.

*Updates

5/19/9 – Two port CCR scenario updated to include recommendation to use continuous replication host name for second port.