Exchange 2010 / 2013 – PAM and the Cluster Core Resources

At any given time, in every database availability group (DAG), there is one member that is responsible for the coordination of database actions across the DAG. This member is known as the Primary Active Manager (PAM). The current PAM can be determined by using Get-DatabaseAvailabilityGroup –Status, as shown below.

 

[PS] C:\>Get-DatabaseAvailabilityGroup -Status -Identity DAG | fl name,primaryActiveManager

Name : DAG
PrimaryActiveManager : MBX-1

 

The mailbox server that is the PAM is always the current owner of the Cluster Core Resource group.

 

[PS] C:\>Get-ClusterGroup -Cluster MBX-1

Name OwnerNode State
---- --------- -----
Available Storage MBX-3 Offline
Cluster Group MBX-1 Online

 

The cluster group may contain several cluster resources. The PAM does not depend on the state of any of the resources in this group, and the PAM role will always be assigned to the node that owns the Cluster Group.

 

The Cluster Group can be moved between members using the cluster management tools. 

 

Windows 2008 R2 / Windows 2012 / Windows 2012 R2:

Command Line:

Cluster.exe cluster <DAGNAME> group “Cluster Group” /moveto:<NODE>

 

Powershell:

Move-ClusterGroup –cluster <DAGNAME> –name “Cluster Group” –node:<NODE>

*Note:  In Windows Server 2012 and Windows Server 2012 R2, Powershell is the preferred way to manage clusters.

 

Each DAG member that does not own the cluster group is a Standby Active Manager (SAM).  When the cluster group is moved between nodes, a notification process detects that the Cluster Group owner has changed. This triggers detection logic to determine the new PAM.  In this example, the cluster group is moved from MBX-1 to MBX-2.

 

PS C:\> Move-ClusterGroup -Name "Cluster Group" -Node MBX-2

Name OwnerNode State
---- --------- -----
Cluster Group MBX-2 Online

 

A review of Microsoft-Exchange-HighAvailability/Operational crimson channel shows the promotion of MBX-2 from SAM to PAM:

 

Log Name: Microsoft-Exchange-HighAvailability/Operational
Source: Microsoft-Exchange-HighAvailability
Date: 8/3/2014 8:37:47 AM
Event ID: 227
Task Category: Role Monitoring
Level: Information
Keywords:
User: SYSTEM
Computer: MBX-2.domain.com
Description:
Active manager configuration change detected. (PreviousRole='SAM', CurrentRole='PAM', ChangeFlags='Role, CurrentPAM', LastError='<none>')

Log Name: Microsoft-Exchange-HighAvailability/Operational
Source: Microsoft-Exchange-HighAvailability
Date: 8/3/2014 8:37:48 AM
Event ID: 111
Task Category: Role Monitoring
Level: Information
Keywords:
User: SYSTEM
Computer: MBX-2.domain.com
Description:
Active manager role changed from 'SAM' to 'PAM'

 

A review of the Microsoft-Exchange-HighAvailability/Operational crimson channel shows the demotion of MBX-1 from PAM to SAM:

 

Log Name: Microsoft-Exchange-HighAvailability/Operational
Source: Microsoft-Exchange-HighAvailability
Date: 8/3/2014 8:37:47 AM
Event ID: 227
Task Category: Role Monitoring
Level: Information
Keywords:
User: SYSTEM
Computer: MBX-1.domain.com
Description:
Active manager configuration change detected. (PreviousRole='PAM', CurrentRole='SAM', ChangeFlags='Role, CurrentPAM', LastError='<none>')

Log Name: Microsoft-Exchange-HighAvailability/Operational
Source: Microsoft-Exchange-HighAvailability
Date: 8/3/2014 8:37:47 AM
Event ID: 111
Task Category: Role Monitoring
Level: Information
Keywords:
User: SYSTEM
Computer: MBX-1.domain.com
Description:
Active manager role changed from 'PAM' to 'SAM'

 

Other servers within the DAG also acknowledge the move but continue to maintain their existing roles:

 

Log Name: Microsoft-Exchange-HighAvailability/Operational
Source: Microsoft-Exchange-HighAvailability
Date: 8/3/2014 8:37:47 AM
Event ID: 227
Task Category: Role Monitoring
Level: Information
Keywords:
User: SYSTEM
Computer: MBX-3.domain.com
Description:
Active manager configuration change detected. (PreviousRole='SAM', CurrentRole='SAM', ChangeFlags='CurrentPAM', LastError='<none>')

 

The Cluster service is also responsible for automatic arbitration of this group.  Automatic arbitration may occur for a number of reasons including: 

 

  • The failure of a member
  • The failure of a resource contained within the Cluster group

 

In most cases, Exchange administrators should not be concerned with the owner of the cluster group or the node designated as the PAM.  This is true even for DAGs that span multiple sites where the PAM may be a node in a distant datacenter.

 

In recent weeks I have fielded several questions from administrators that are concerned with which member owns the PAM. Questions like:

 

  • Should I set a preferred owner on the cluster group so that nodes in my primary datacenter are preferred over my disaster recovery datacenter?
  • How do I prevent a server in the disaster recovery datacenter from becoming the PAM?
  • Can I pause my nodes in cluster to prevent them from becoming the PAM?
  • Should I remove possible owner on the cluster name to prevent it from coming online on a server in my disaster recovery datacenter?

 

All of these questions require modifying properties of the cluster core resource group.  By default Exchange establishes the desired settings on the cluster core resource group.  Modifying these settings is typically not necessary and can sometimes cause undesired results.  For example, recently a customer paused all of the members in a disaster recovery datacenter to prevent those servers from becoming the PAM.  This worked very well in preventing arbitration of the cluster core resource group to these nodes, until instability of the Cluster service in the primary datacenter resulted in no members being able to take ownership the cluster core resources.  In this instance the PAM was lost, and coordination of database activities across the DAG failed. 

 

The Cluster service is designed to allow the cluster group to arbitrate freely across nodes.  Attempts to modify the failover behavior, or prevent failure between nodes, can yield undesired results and potential instability.  As a result, we recommend that you leave the default out-of-box settings intact, unless you are directed by Microsoft Customer Services and Support to change them.