Exchange 2010: Remove-databaseavailabilitygroupserver–configurationOnly does not evict the member from the cluster.

Administrators may encounter conditions where DAG members cannot be gracefully removed from a database availability group.  For example, a member server may have encountered an unrecoverable failure or the server may need to be removed from the DAG in order to perform a server recovery.


In order to account for these and similar conditions the remove-databaseavailabilitygroupserver –configurationOnly command exists.  This command, when utilized, simply removes the member from the Database Availability Groups Active Directory object.


Here is an example…


Using get-databaseavailabilitygroup –status | fl name,servers,operationalservers the membership of the DAG can be verified.  In this example the only operational server is MBX-1 since that is the only server currently running in the DAG.


[PS] C:\>Get-DatabaseAvailabilityGroup DAG -status | fl name,Servers,OperationalServers

Name               : DAG
Servers            : {MBX-1, MBX-2}
OperationalServers : {MBX-1}


Using the remove-databaseavailabilitygroupserver –configurationOnly command a DAG member can be removed.


[PS] C:\>Remove-DatabaseAvailabilityGroupServer -Identity DAG -MailboxServer MBX-2 -ConfigurationOnly

Are you sure you want to perform this action?
Removing Mailbox server "MBX-2" from database availability group "DAG".
[Y] Yes [A] Yes to All [N] No [L] No to All [?] Help (default is "Y"): a


The results of the command can be verified using get-databaseavailabilitygroup –status | fl name,servers,operationalservers:


[PS] C:\>Get-DatabaseAvailabilityGroup DAG -status | fl name,Servers,OperationalServers

Name : DAG
Servers : {MBX-1}
OperationalServers : {MBX-1}


When a server is removed from the DAG in this manner it is not evicted from the corresponding cluster.  You can verify cluster membership using the built in cluster commands.  Here is an example from this test:


( Windows 2008 / Windows 2008 R2 )


[PS] C:\>cluster.exe node
Listing status for all available nodes:

Node           Node ID Status
————– ——- ———————
MBX-1                1 Up
MBX-2                2 Down

( Windows 2008 R2 )

[PS] C:\>Import-Module FailoverClusters

[PS] C:\>Get-ClusterNode

Name                                State
—-                                —–
mbx-1                                  Up
mbx-2                                Down


In general this issue surfaces when administrators complete a server rebuild operation and note that the rebuilt node cannot be added back to the cluster because it already exists in the cluster.  Here is an example:


[PS] C:\>Add-DatabaseAvailabilityGroupServer –identity DAG –mailboxServer MBX-2

WARNING: The operation wasn’t successful because an error was encountered. You may find more details in log file
A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Node mbx-2 is already joined to a cluster. [Server:]
    + CategoryInfo          : InvalidArgument: (:) [Add-DatabaseAvailabilityGroupServer], DagTaskOperationFailedException
    + FullyQualifiedErrorId : D05F37CD,Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer



When using the remove-databaseavailabilitygroupserver –configurationOnly administrators must remove the node from the cluster.  This can be accomplished through two methods:


( Windows 2008 / Windows 2008 R2 )


Administrators may utilize Failover Cluster Manager.  After connecting to the cluster servicing the Database Availability Group the nodes hive can be expanded.  The administrator can right click on the node that was removed –> select more actions –> evict




( Windows 2008 R2 )

[PS] C:\>Import-Module FailoverClusters

[PS] C:\>Remove-ClusterNode MBX-2

Are you sure you want to evict node mbx-2?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): y
Remove-ClusterNode : The cluster node ‘MBX-2’ was evicted from the cluster, but was not fully cleaned up.  Please see the Failover Clustering application event log on node MBX-2 for more information.
    The RPC server is unavailable
At line:1 char:19
+ Remove-ClusterNode <<<<  MBX-2
    + CategoryInfo          : NotSpecified: (:) [Remove-ClusterNode], ClusterCmdletException
    + FullyQualifiedErrorId : Remove-ClusterNode,Microsoft.FailoverClusters.PowerShell.RemoveClusterNodeCommand


(Note:  The RPC error is expected as the command attempts to cleanup the local cluster configuration on the node but the node is not accessible)


After cleaning up the cluster configuration the administrator can run set-databaseavailabilitygroup –identity <DAGNAME> to ensure the appropriate cluster configuration is utilized.

Comments (8)

  1. FT says:

    Great article! Thanks for taking the time to put this together and for sharing it with the rest of the Exchange Community!

    Best Regards,


  2. Kevin says:

    Excellent Article

    I have DAG with 4 nodes on the primary site and 3 nodes on the dr site
    I need to shutdown all 4 nodes on the primary site at the same time for maintenance
    I assume that there is a way of doing that without having to use classic DAG Disaster Recovery (and haing the whole DAG going down) ?

    If I use the procedure described in your article to evict the 4 servers of the primary site , will it be easy to add them back (reseed all databases… ) ?


  3. Megan says:

    Thanks for this, really saved my bacon!

  4. alfredo bontigao rance jr says:

    go back to profill

  5. Steven says:

    Great article! Wondering if you could comment on a scenario. Take a server that’s going to be down for a while, but isn’t going to be otherwise rebuilt, maybe we need to wait some time for parts or something. The server is part of a 4 node DAG and in the
    primary data center, so for the long duration of the outage we don’t have Quorum in the primary data center alone, a WAN outage could bring down production! I add another server to the DAG to get an extra vote, but that switches the quorum model to "Majority
    Node" effectively removing the witness vote. The issue is I still don’t have Quorum in the primary data center for the duration of the outage, and would have to add yet another server to get it. There are 2 questions I garnish from this:

    1) Is there any way to force the Quorum model to "Majority Node & Witness" regardless of node count?
    2) If no to number 1, does adding the second server at least give me a quick way to re-establish quorum by using Stop-Database AvailabilityGroup -MailboxServer or is there another take like using Remove-DatabaseAvailabilityGroupServer -Configuration Only as
    you’ve mentioned above.

    Let me know what you think. Thanks

  6. TIMMCMIC says:


    This is always a tough situation. There is no way to force the quorum model to change as Exchange enforces it and it’s not supported to do so outside of Exchange.

    Stop and remove will not help much here overall, as it would just do what adding or removing the node is.

    You may have to consider adding additional voter nodes.


  7. KC Hemdanie says:

    Consider keeping a dormant File Share Witness virtual server at primary. You can always clone the FSW server and keep it dormant so in the event of a member/network issue you can bring it up on line fast. I have 2 DAGS a 14 member and a 4 member DAG. evenly
    split between 2 datacenter w/redundant GB networks between them. My FSW sits at a 3rd site (HQ) that has redundant separate paths to both datacenters… I also have a dormant clone of the FSW in each of the datacenter as well…. both datacenters act as primary
    w/ automatic failover. I guess you can call it a stretched DAG across the datacenters… Microsoft says it is not a design supported in Exchange 2010 but it is supported in 2013. If you can afford a spare VM for FSW… then use it as a dormant standby in case
    you lose network the DR site.

  8. TIMMCMIC says:

    @KC. …

    Just to clarify – I have never seen an official support policy for a dormant file share witness server. I personally would not recommend this approach. There are potential issues with this – both with the Kerberos security context of the VM up to the file share
    witness and the paxos information potentially contained in the file share witness.

    Those that are concerned about witness redundancy should really either consider a legitimate third site witness keeping an additional witness server available and adjusting with set-databaseavailabilitygroup as necessary.