Continuous Replication Hostnames fail to create or function correctly with Exchange 2007 SP3 Cluster Continuous Replication (CCR) on Windows 2008 R2

  • Article

Exchange 2007 SP3 adds the support for utilizing Windows 2008 R2 servers. 

In Exchange 2007 Cluster Continuous Replication (CCR) installations, all log shipping activity by default occurs over the “public” cluster interface. When administrators desire to have log shipping activities occur over a “private” network or desire to implement multiple replication paths between nodes, continuous replication hostnames can be utilized.

More information on Exchange 2007 CCR clusters and continuous replication hostnames can be found at https://technet.microsoft.com/en-us/library/bb124521(EXCHG.80).aspx.

Prior to implementing a continuous replication host name the get-clusteredservermailboxstatus commandlet can be utilized to see the current names services replication. Here is a sample output from a cluster not configured to utilize continuous replication hostnames.

Identity : MBX-3
ClusteredMailboxServerName : MBX-3.domain.com
State : Online
OperationalMachines : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources : {}
OperationalReplicationHostNames : {node-1, node-2}
FailedReplicationHostNames : {}
InUseReplicationHostNames : {node-1, node-2}
IsValid : True
ObjectState : Unchanged

After establishing the pre-requisites necessary to utilize continuous replication hostnames, the hostnames creation is performed using the enable-continuousreplicationhostname shell command. (https://technet.microsoft.com/en-us/library/bb690985(EXCHG.80).aspx)

When attempting to enable a replication hostname on a Windows 2008 R2 cluster, the following error may be displayed in the management shell.

[PS] C:\>Enable-ContinuousReplicationHostName -TargetMachine Node-1 -HostName Node-1-Repl-A -IPv4Address 10.0.1.3

Confirm
Are you sure you want to perform this action?

Enabling continuous replication host name "Node-1-Repl-A".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help
(default is "Y"):a
Enable-ContinuousReplicationHostName : Enable-ContinuousReplicationHostNameNetw
ork configuration could not be completed.
At line:1 char:37
+ Enable-ContinuousReplicationHostName <<<< -TargetMachine Node-1 -HostName Node-1-Repl-A -IPv4Address 10.0.1.3
+ CategoryInfo : InvalidOperation: (:) [Enable-ContinuousReplicat
ionHostName], NetworkConfigException
+ FullyQualifiedErrorId : C3F1320,Microsoft.Exchange.Management.SystemConf
igurationTasks.EnableContinuousReplicationHostName

When reviewing Failover Cluster Manager, the replication host name group containing the correct network name and ipv4 address appear to have been created successfully.

image

image

Although the continuous replication hostname group was created, reviewing get-clusteredservermailboxstatus indicates the name is not being utilized by the replication service on the cluster.

Identity : MBX-3
ClusteredMailboxServerName : MBX-3.domain.com
State : Online
OperationalMachines : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources : {}
OperationalReplicationHostNames : {node-1, node-2}
FailedReplicationHostNames : {}
InUseReplicationHostNames : {node-1, node-2}
IsValid : True
ObjectState : Unchanged

When the replication service first starts up <or> the configuration time expires the replication service enumerates all network names on the cluster to determine which are valid endpoints for log shipping. This is initially based on two cluster private properties stamped on each name, MSExchange_NetName and MSExchange_UseNetworkForLogCopying. Each of these should have a value of 1 on a network name utilized as a continuous replication host name.

Listing private properties for 'Network Name (Node-1-Repl-A)':

T Resource Name Value

-- -------------------- ------------------------------ -----------------------

BR Network Name (Node-1-Repl-A) ResourceData 01 00 00 00 ... (260 bytes)

DR Network Name (Node-1-Repl-A) StatusNetBIOS 0 (0x0)

DR Network Name (Node-1-Repl-A) StatusDNS 0 (0x0)

DR Network Name (Node-1-Repl-A) StatusKerberos 0 (0x0)

SR Network Name (Node-1-Repl-A) CreatingDC \\DC-1.domain.com

FTR Network Name (Node-1-Repl-A) LastDNSUpdateTime 7/11/2010 2:26:26 PM

SR Network Name (Node-1-Repl-A) ObjectGUID 5adc38b3281a004788f2a3e27ae7a0ce

S Network Name (Node-1-Repl-A) Name NODE-1-REPL-A

S Network Name (Node-1-Repl-A) DnsName Node-1-Repl-A

D Network Name (Node-1-Repl-A) RemapPipeNames 0 (0x0)

D Network Name (Node-1-Repl-A) HostRecordTTL 1200 (0x4b0)

D Network Name (Node-1-Repl-A) RegisterAllProvidersIP 0 (0x0)

D Network Name (Node-1-Repl-A) PublishPTRRecords 0 (0x0)

D Network Name (Node-1-Repl-A) TimerCallbackAdditionalThreshold 5 (0x5)

D Network Name (Node-1-Repl-A) MSExchange_NetName 1 (0x1)

D Network Name (Node-1-Repl-A) RequireDNS 1 (0x1)

D Network Name (Node-1-Repl-A) MSExchange_UseNetworkForLogCopying 1 (0x1)

On the surface it would appear that there is nothing preventing this name from operating correctly as a continuous replication host name. After performing some internal tracing it was determined that the replication service is also implementing another check on a network name resource to ensure that it can be satisfactorily utilized for replication – is Kerberos enabled for the network name. The replication service performs this check by reviewing a private property of a network name resource – requirekerberos and ensuring it has a value of 1.

In Windows 2003 network name resources could be enabled for Kerberos at the administrators discretion. In Windows 2008 and Windows 2008 R2 all network names must be Kerberos enabled. In Windows 2008 requireKerberos is a valid private property and can be programatically set. In Windows 2008 R2 the requireKerberos property has been deprecated and can be no longer be programmatically set. Without the requireKerberos property in Windows 2008 R2 the enable-continuousreplicationhostname commandlet fails with the previously documented error. 

To work around this issue and allow the replication host names created with the enable-continuousreplicationhostname command to function the following steps can be performed:

  • Using the Exchange Managment Shell invoke the enable-continuousreplicationhostname command. Allow the command to create the resource group, network name, and IPv4 resource. 
  • Verify with Failover Cluster Manager that the resource group, network name, and IPv4 resource were created and are online.
  • Manually set requireKerberos utilizing either cluster.exe or Failover Cluster Powershell extensions (preferred)
    • Cluster.exe
      • Set the requirekerberos key.
        • Cluster.exe <clusterFQDN> res "<Network Name> /priv requirekerberos=1:DWORD
        • Example: cluster.exe cluster cluster-1.domain.com res “Network Name (Node-1-Repl-A)” /priv requirekerberos=1:DWORD
        • Note that requirekerberos is all lowercase.
      • Take offline and online the continuous replication hostname group.
        • Cluster.exe <clusterFQDN> group <Group> /offline
        • Example: cluster.exe cluster.domain.com group “Node-1-Repl-A_group” /offline
        • Cluster.exe <clusterFQDN group <Group> /online
        • Example: cluster.exe cluster.domain.com group “Node-1-Repl-A_group” /online
      • Restart the replication service
        • net stop msexchangerepl
        • net start msexchangerepl
    • PowerShell
      • Import the failover cluster powershell extensions.
        • Import-Module FailoverClusters
      • Set the requirekerberos key.
        • Get-ClusterResource <Network Name> | Set-ClusterParameter requirekerberos 1
        • Example: Get-ClusterResource “Network Name (Node-1-Repl-A)” | Set-ClusterParameter –create requirekerberos 1
        • Node that requirekerberos is all lowercase.
      • Take offline and online the continuous replication hostname group.
        • Stop-ClusterGroup –cluster <ClusterFQDN> –Name <Group>
        • Example: Stop-ClusterGroup –cluster Cluster.domain.com –Name Node-1-Repl-A_group
        • Start-ClusterGroup –cluster <ClusterFQDN> –Name <Group>
        • Example: Start-ClusterGroup –cluster Cluster.domain.com –Name Node-1-Repl-A_group
      • Restart the replication service.
        • Stop-Service msexchangerepl
        • Start-Service msexchangerepl

At this time you can utilize either cluster.exe or powershell to verify that the requirekerboros key has been created with a value of 1.

Cluster.exe <clusterFQDN> res <Network Name> /priv --> Cluster.exe cluster.domain.com res “Network Name (Node-1-Repl-A)” /priv

Listing private properties for 'Network Name (Node-1-Repl-A)':

T Resource Name Value

-- -------------------- ------------------------------ -----------------------

BR Network Name (Node-1-Repl-A) ResourceData 01 00 00 00 ... (260 bytes)

DR Network Name (Node-1-Repl-A) StatusNetBIOS 0 (0x0)

DR Network Name (Node-1-Repl-A) StatusDNS 0 (0x0)

DR Network Name (Node-1-Repl-A) StatusKerberos 0 (0x0)

SR Network Name (Node-1-Repl-A) CreatingDC \\DC-1.domain.com

FTR Network Name (Node-1-Repl-A) LastDNSUpdateTime 7/11/2010 2:26:26 PM

SR Network Name (Node-1-Repl-A) ObjectGUID 5adc38b3281a004788f2a3e27ae7a0ce

S Network Name (Node-1-Repl-A) Name NODE-1-REPL-A

S Network Name (Node-1-Repl-A) DnsName Node-1-Repl-A

D Network Name (Node-1-Repl-A) RemapPipeNames 0 (0x0)

D Network Name (Node-1-Repl-A) HostRecordTTL 1200 (0x4b0)

D Network Name (Node-1-Repl-A) RegisterAllProvidersIP 0 (0x0)

D Network Name (Node-1-Repl-A) PublishPTRRecords 0 (0x0)

D Network Name (Node-1-Repl-A) TimerCallbackAdditionalThreshold 5 (0x5)

D Network Name (Node-1-Repl-A) MSExchange_NetName 1 (0x1)

D Network Name (Node-1-Repl-A) RequireDNS 1 (0x1)

D Network Name (Node-1-Repl-A) MSExchange_UseNetworkForLogCopying 1 (0x1)

D Network Name (Node-1-Repl-A) requirekerberos 1 (0x1)

Get-ClusterResource <NAME> | Get-ClusterParameter

Object Name Value Type
------ ---- ----- ----
Network Name (No... Name NODE-1-REPL-A String
Network Name (No... DnsName Node-1-Repl-A String
Network Name (No... RemapPipeNames 0 UInt32
Network Name (No... HostRecordTTL 1200 UInt32
Network Name (No... RegisterAllProvi... 0 UInt32
Network Name (No... PublishPTRRecords 0 UInt32
Network Name (No... TimerCallbackAdd... 5 UInt32
Network Name (No... MSExchange_NetName 1 UInt32
Network Name (No... RequireDNS 1 UInt32
Network Name (No... MSExchange_UseNe... 1 UInt32
Network Name (No... requirekerberos 1 UInt32
Network Name (No... ResourceData {1, 0, 0, 0, 118... ByteArray
Network Name (No... StatusNetBIOS 0 UInt32
Network Name (No... StatusDNS 0 UInt32
Network Name (No... StatusKerberos 0 UInt32
Network Name (No... CreatingDC \\DC-1.domain...... String
Network Name (No... LastDNSUpdateTime 7/11/2010 9:26:2... DateTime
Network Name (No... ObjectGUID 5adc38b3281a0047... String            

By restarting the replication service after setting this key the replication services configuration is immediately updated. At this time the replication service should detect and begin to utilize the replication hostnames created. This can be verified using the get-clusteredservermailboxstatus commandlet.

Identity : MBX-3
ClusteredMailboxServerName : MBX-3.exchange.msft
State : Online
OperationalMachines : {NODE-1 <Active>, Node-2 <Quorum Owner>}
FailedResources : {}
OperationalReplicationHostNames : {node-1-repl-a, node-1, node-2}
FailedReplicationHostNames : {}
InUseReplicationHostNames : {node-1-repl-a, node-2}
IsValid : True
ObjectState : Unchanged

At this time we are investigating a fix that does not require a workaround.  As changes occur I will update this blog.