Exchange 2010 SP1: StartDagServerMaintenance.ps1 fails when a server contains databases with a single copy.

In Exchange 2010 Service Pack 1 we introduced some new DAG management scripts.  These scripts can be found in the Exchange Server installation directory \ scripts.  (This is usually c:\Program Files\Microsoft\Exchange Server\v14\scripts).

 

One of the scripts introduced is the StartDagServerMaintenance.ps1 script.  More information on this script can be found at:

 

https://technet.microsoft.com/en-us/library/ff625233.aspx

https://technet.microsoft.com/en-us/library/dd298065.aspx

 

When administrators utilize this script the following actions are being taken:

1)  All database copies are moved to another server in the DAG based on the selection of the next best copy.

2)  If the cluster core resources are owned on the node the resources are arbitrated to a different DAG member (thereby moving the Primary Active Manager functionality to another node).

3)  The DatabaseCopyAutoActivationPolicy property of the mailbox server is set to a value of BLOCKED thereby preventing the DAG member from receiving or activating database copies.

4)  The individual database copies hosted on the DAG member are activation suspended.

5)  The node is paused within the cluster service preventing the cluster core resources from arbitrating to the node (and thereby preventing the node from becoming the Primary Active Manager).

 

When utilizing a DAG it is not necessary to replicate all databases that exist on DAG members.  It is not uncommon to have standalone databases (databases that are on a DAG member but not replicated to another member) present on a member where the StartDagServerMaintenance.ps1 script will be utilized.  Unfortunately when utilizing the script in its current form in this configuration the script fails to complete its tasks and cannot completely put the node into maintenance mode.   (Only databases are successfully moved off the member).

 

The administrator may note the following when executing the script on a member that contains a single database copy:

 

[PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>.\StartDagServerMaintenance.ps1 -serverName DAG-1

The following objects are hosted by 'DAG-1', before attempting to move them off: `n(Primary Active Manager=DAG-1) (Mailbox='Discovery Search Mailbox', Reason='Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database. ) (Mailbox='Journal Internal', Reason='Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database. ) (Mailbox='MicrosoftExchange Approval Assistant', Reason='Arbitration Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database.) (Database='DAG-DB0', Reason='Copy is active'))

Write-Error : The following objects are still hosted by 'DAG-1', even after attempting to move them off: `n(Mailbox='Discovery Search Mailbox', Reason='Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database. ) (Mailbox='Journal Internal', Reason='Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database. ) (Mailbox='Microsoft Exchange Approval Assistant', Reason='Arbitration Mailbox is hosted on 'DAG-1-DB0', which is not a replicated database. ))
At C:\Program Files\Microsoft\Exchange Server\V14\Scripts\StartDagServerMaintenance.ps1:216 char:16
+ write-error <<<< ($StartDagServerMaintenance_LocalizedStrings.res_0014 -f ( PrintCriticalMailboxResourcesOutput($criticalMailboxResources)),$shortServerName) -erroraction:stop
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Microsoft.PowerShell.Commands.WriteErrorCommand

 

If an administrator encounters this condition the following process can be utilized to place the DAG member into maintenance mode.  (In our example server DAG-1 in the DAG named “DAG” [pretty creative eh?] is the server we will be placing in maintenance mode)

 

1)  Execute a get-mailboxdatabasecopystatus * and verify that at least one other non-lagged copy of each replicated database is healthy.

 

[PS] C:\>Get-MailboxDatabaseCopyStatus *

Name Status CopyQueue ReplayQueue LastInspectedLogTime ContentIndex
Length Length State
---- ------ --------- ----------- -------------------- ------------
DAG-1-DB0\DAG-1 Mounted 0 0 Healthy
DAG-DB0\DAG-1 Mounted 0 0 Healthy
DAG-DB1\DAG-1 Healthy 0 0 7/13/2011 8:22:55 AM Healthy
DAG-2-DB0\DAG-2 Mounted 0 0 Healthy
DAG-DB1\DAG-2 Mounted 0 0 Healthy
DAG-DB0\DAG-2 Healthy 0 4 7/13/2011 8:48:34 AM Healthy
DAG-DB0\DAG-3 Healthy 0 147 7/13/2011 8:48:34 AM Healthy
DAG-DB1\DAG-3 Healthy 0 140 7/13/2011 8:22:55 AM Healthy
DAG-DB0\DAG-4 Healthy 0 409 7/13/2011 8:48:34 AM Healthy
DAG-DB1\DAG-4 Healthy 0 307 7/13/2011 8:22:55 AM Healthy
MBX-1-DB0\MBX-1 Mounted 0 0 Healthy
MBX-1-RDB\MBX-1 Mounted 0 0 Healthy

2)  Execute a move of all active database copies off the server.  This can be done with the command move-activemailboxdatabase –server <MaintenanceServer>  (Note:  No target server is specified which means the next best copy will be automatically selected for activation)

 

[PS] C:\>Move-ActiveMailboxDatabase -Server DAG-1 -Confirm:$FALSE

Identity ActiveServerAtS ActiveServerAtE Status NumberOfLogsLost RecoveryPoint MountStatus MountStatus
tart nd Objective AtMoveStart AtMoveEnd
-------- --------------- --------------- ------ ---------------- ------------- ----------- -----------
DAG-1-DB1 dag-1 dag-1 Warning Mounted Mounted
DAG-1-DB0 dag-1 dag-1 Warning Mounted Mounted
DAG-DB0 dag-1 dag-2 Succeeded 0 7/13/2011 8:5 Mounted Mounted
3:24 AM
WARNING: An Active Manager operation failed. Error: The database action failed. Error: You cannot perform a switchover
operation on database 'DAG-1-DB1' because the database is not configured for replication.. [Database: DAG-1-DB1,
Server: DAG-1.domain.com]
WARNING: An Active Manager operation failed. Error: The database action failed. Error: You cannot perform a switchover
operation on database 'DAG-1-DB0' because the database is not configured for replication.. [Database: DAG-1-DB0,
Server: DAG-1.domain.com]

3)  Move the cluster core resources to another node within the DAG.  This can be accomplished using the command cluster.exe <DAGFQDN> group “Cluster Group” /moveto:<NODE>

 

[PS] C:\>cluster DAG.domain.com group "Cluster Group" /moveto:DAG-2

Moving resource group 'Cluster Group'...

Group Node Status
-------------------- --------------- ------
Cluster Group DAG-2 Online

4) Pause the node within the cluster.  This can be done utilizing the command cluster.exe <DAGFQDN> node <NODENAME> /pause

 

[PS] C:\>cluster DAG.domain.com node DAG-1 /pause

Pausing node 'DAG-1'...

Node Node ID Status
-------------- ------- ---------------------
DAG-1 1 Paused

5) Set the DatabaseCopyAutoActivationPolicy of the server to BLOCKED.  This can be done using the command set-mailboxserver –identity <DAGMember> –databasecopyautoactivationpolicy:BLOCKED

 

[PS] C:\>Set-MailboxServer -Identity DAG-1 -DatabaseCopyAutoActivationPolicy:BLOCKED

6) Suspend all individual copies for activation.  This can be done using the command get-mailboxdatabasecopystatus *\<DAGMember> | suspend-mailboxdatabasecopy –activationOnly:$TRUE

 

[PS] C:\>Get-MailboxDatabaseCopyStatus *\DAG-1 | Suspend-MailboxDatabaseCopy -ActivationOnly:$TRUE
Database "DAG-1-DB0\DAG-1" has only one copy. This task is supported only for databases that have more than one copy.
+ CategoryInfo : InvalidOperation: (DAG-1-DB0:ADObjectId) [Suspend-MailboxDatabaseCopy], InvalidOperation
Exception
+ FullyQualifiedErrorId : 7325D1AB,Microsoft.Exchange.Management.SystemConfigurationTasks.SuspendDatabaseCopy

Confirm
Are you sure you want to perform this action?
Suspending activation of mailbox database copy "DAG-DB0" on server "DAG-1".
[Y] Yes [A] Yes to All [N] No [L] No to All [?] Help (default is "Y"): a

 

At this time it should be safe for the administrator to perform DAG server maintenance.  When the maintenance is complete the script StopDagServerMaintenance.ps1 can be utilized to take the DAG member out of maintenance mode.