Kerberos authentication, dns suffix lists, the replication service, and Exchange management commandlets…

When using any form of multi-machine Exchange 2007 replication (CCR / SCR) Kerberos authentication is very important.  We leverage the rights of Exchange server machine accounts for several functions including the ability to replicate log files and utilize the remote registry service for commandlets.

Some Background…

In terms of the replication service we copy logs between CCR and SCR servers using SMB file shares.  These shares are created by the replication service.  Permissions to access these shares are derived by assigning the share permission READ to the Exchange Servers group.

Note:  We use a very restrictive read heuristic which does not fully equal the same read permission as set through the GUI so you’ll have to trust me here – the groups effective share permission is read.

In the Exchange Servers group we automatically place the machine account for each Exchange server installed. 

By adding the Exchange servers machine account to the Exchange Servers group, and adding the Exchange Servers group with appropriate permissions, we’ve effectively allowed the machine accounts read access to the shares.  The replication service, which runs under the local system security context, can then access the shares to pull log files.

The Issue…

It is becoming more common in environments to no longer see WINS installations.  It is still a requirement of the product though that short name resolution work.  Many administrators address this issue by using a DNS suffix list.  You can set the DNS suffix list on the advanced settings of TCP/IP.

When a short name resolution request is made, the domains are appended in order as specified on the list.  If the name can be found in the DNS zone of the name as appended, then this combination is returned as the fully qualified domain name.

For example, if I have a host Server1 that is registered only in the DNS namespace exchange.msft, when I issue a ping request to Server1 the first domain appended is external.exchange.msft.  Since the machine is not registered in that dns domain, the second address is appended, in this case exchange.msft.  With the machine registered in this domain, a successful name query response is received and the ping will continue successfully to server1.exchange.msft.

An ipconfig /all will display the appended DNS suffixes and their order.

In many circumstances the machine will only resolve in a single DNS domain.  If this is the case then it will not affect keberos authentication.  Where an issue occurs is where the machine resolves in more then one domain, and the domain it resolves it does not match the active directory domain (which in this case is what is registered on the service principle name records) of the machine account.  Lets look at an example of how this can be an issue.

In this example the machine Server1 is registered in the dns domain external.exchange.msft and exchange.msft.  My active directory DNS name space for the domain that the machine is a member of is exchange.msft.

My append dns suffixes in this order has the following list:

external.exchange.msft

exchange.msft

domain.com

When the replication service attempts to copy a log file it begins the authentication process to the share.  The first step in this process is obtaining a kerberos ticket so we can leverage the permissions of the machine account (local system) for share access.  The first name in the suffix list is appended, and a successful name resolution occurs.  In this case the fully qualified domain name is believed to be Server1.external.exchange.msft.  At this time the kerberos key distribution center is contacted, and a ticket is issued for Server1.external.exchange.msft.  The next step is to access the share presenting this kerberos ticket.  At this time an access denied is received to the share, and logs cannot be copied. 

The reason the access denied is returned is that the service principle names stamped on the machine account in active directory for the server does not include Server1.external.exchange.msft, it only includes Server1.exchange.msft (the AD domain name).  You can see the SPNs registered on the server by doing an LDP dump of the computer object in the active directory domain container.  Here is an example:

servicePrincipalName (4): MSServerClusterMgmtAPI/2008-NODE1; MSServerClusterMgmtAPI/2008-Node1.exchange.msft; HOST/2008-Node1; HOST/2008-Node1.exchange.msft;

The issue in this case is easily corrected.  To correct the issue, change the appended DNS suffix list to use the active directory domain first.  For example:

exchange.msft

external.exchange.msft

domain.com

With the updated DNS suffix list the server name determined is server1.exchange.msft.  This name matches the entries of the service principle name and authentication can occur successfully and therefore log replication can occur without issues.

Other functions besides log replication can be impacted by the appended DNS suffix list.  For example, certain commandlets such as get-storagegroupcopystatus and update-storagegroupcopy leverage the rights of the local system to access the remote registry service.  These commandlets can also suffer access denied conditions as authenticated remote registry connections between servers can fail. 

Here is a sample of the error text from a failed get-storagegroupcopystatus:

Microsoft Exchange Replication service RPC failed :
Microsoft.Exchange.Rpc.RpcException: Error e0434f4d from cli_GetCopyStatusEx
at Microsoft.Exchange.Rpc.Cluster.ReplayRpcClient.GetCopyStatusEx(Guid[]
sgGuids, RpcStorageGroupCopyStatus[]& sgStatuses)
at
Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.InternalGetCopyStatus(Strin
g serverName, Guid[] sgGuids, RpcStorageGroupCopyStatus[]& sgStatuses, Int32
serverVersion)
at
Microsoft.Exchange.Cluster.Replay.RpcCopyStatusInfo.GetMergedStatusResults()
at
Microsoft.Exchange.Management.SystemConfigurationTasks.GetStorageGroupCopyStatus.Pre
pareStatusEntryFromRpc(Boolean fCcr, Server server, StorageGroup storageGroup,
StorageGroupCopyStatusEntry& entry)"

The moral of the story…

Replication and commandlet issues on Exchange servers can be avoided when using appended dns suffixes list but ensuring that the active directory DNS domain is the first to be appended.