Restore-StorageGroupCopy –standbymachine requires use of –force in order to complete successfully when activating a Standby Continuous Replication target in Exchange 2007 SP1.

When attempt to run restore-storagegroupcopy on an SCR target you may receive the following error (note this is with verbose output):

[PS] C:\Windows\System32>Restore-StorageGroupCopy MBX-4\MBX-4-SG1 -StandbyMachine MBX-3 -Verbose > output.txt
VERBOSE: Restore-StorageGroupCopy : Beginning processing.
VERBOSE: Restore-StorageGroupCopy : Searching objects "MBX-4\MBX-4-SG1" of type "StorageGroup" under the root "$null".
VERBOSE: Restore-StorageGroupCopy : Previous operation run on domain controller 'DC-3.domain.com.
VERBOSE: Restore-StorageGroupCopy : Processing object "MBX-4\MBX-4-SG1".
VERBOSE: Restoring Storage Group Copy "MBX-4\MBX-4-SG1".
WARNING: Failed to copy remaining log files (through Exx.log)during
Restore-StorageGroupCopy operation for storage group copy (MBX-4-SG1). Failure reason: Unable to move a log file from the inspector directory to the log directory for storage group 9b5be25a-1c2b-48a0-9087-2819d2887001|Standby. Old path: f:\MBX-4\MBX-4-SG1-Logs\E00.log. New path: f:\MBX-4\MBX-4-SG1-Logs\IgnoredLogs\E00OutofDate\2009-02-17T14-42-53E00.log.
.
Restore-StorageGroupCopy : Failed to copy the last log files for storage group 'MBX-4-SG1'. Use the Force option if you want to restore the storage group despite the data loss.
At line:1 char:25
+ Restore-StorageGroupCopy <<<< MBX-4\MBX-4-SG1 -StandbyMachine MBX-3 -Verbose
VERBOSE: Restore-StorageGroupCopy : Ending processing.

When reviewing the application log of the server, you will note the following events:

Log Name: Application
Source: MSExchangeRepl
Date: 2/17/2009 9:42:53 AM
Event ID: 2013
Task Category: Service
Level: Information
Keywords: Classic
User: N/A
Computer: MBX-3.domain.com
Description:
The replication instance for storage group MBX-4\MBX-4-SG1 found an invalid copy of log file f:\MBX-4\MBX-4-SG1-Logs\inspector\E00.log. The log file has been moved to f:\MBX-4\MBX-4-SG1-Logs\IgnoredLogs\InspectionFailed\2009-02-17T14-42-53E00.log. Reason: The log file has failed inspection. It is inappropriate for replay..

Log Name: Application
Source: MSExchangeRepl
Date: 2/17/2009 9:42:53 AM
Event ID: 2089
Task Category: Action
Level: Information
Keywords: Classic
User: N/A
Computer: MBX-3.domain.com
Description:
The Restore-StorageGroupCopy operation on MBX-4\MBX-4-SG1 failed to complete with error code Microsoft.Exchange.Management.Tasks.RestoreSCRFailedToCopyLastLog: Failed to copy the last log files for storage group 'MBX-4-SG1'. Use the Force option if you want to restore the storage group despite the data loss..

In the output of the management shell command the user is advised to use the –FORCE option to complete this operation.  There are certain circumstances where this is necessary, this is not one of them.  The reason this should not be necessary is that the source is fully available so all logs not yet replicated should be available for copy when running restore-storagegroupcopy.

The issue here arises from a bug.  We will attempt to copy the last log up to 6 times.  In this case, regardless of our success or failure, we continue to copy the logs until we reach the max attempts and then display a failure.  In all actuality the command completed successfully.  Let’s take a look at that.

If you review the log file directory on the SCR target, you will see that the ENN.log is actually present in the log file folder.

 

image

 

Using eseutil /ml <log>, we can dump the header information for the log file on the SCR TARGET log directory.

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server

Version 08.01

Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...

      Base name: e00
Log file: e00.log
lGeneration: 2959 (0xB8F)
Checkpoint: NOT AVAILABLE
creation time: 02/17/2009 09:26:41
prev gen time: 02/17/2009 09:26:40
Format LGVersion: (7.3704.12)
Engine LGVersion: (7.3704.12)
Signature: Create time:01/08/2009 10:39:04 Rand:47957569 Computer:
Env SystemPath: e:\MBX-4\MBX-4-SG1-System\
Env LogFilePath: f:\MBX-4\MBX-4-SG1-Logs\
Env Log Sec size: 512
Env (CircLog,Session,Opentbl,VerPage,Cursors,LogBufs,LogFile,Buffers)
( off, 552, 27600, 15960, 27600, 2048, 2048,2000000000)
Using Reserved Log File: false
Circular Logging Flag (current file): off
Circular Logging Flag (past files): off

      Last Lgpos: (0xb8f,8,30)

Integrity check passed for log file: e00.log

Operation completed successfully in 0.63 seconds.

The most important item in this output to us is the signature. 

Using eseutil /ml <log>, we can dump the header information for the log file on the SCR SOURCE log directory.

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server

Version 08.01

Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...

      Base name: e00
Log file: e00.log
lGeneration: 2959 (0xB8F)
Checkpoint: NOT AVAILABLE
creation time: 02/17/2009 09:26:41
prev gen time: 02/17/2009 09:26:40
Format LGVersion: (7.3704.12)
Engine LGVersion: (7.3704.12)
Signature: Create time:01/08/2009 10:39:04 Rand:47957569 Computer:
Env SystemPath: e:\MBX-4\MBX-4-SG1-System\
Env LogFilePath: f:\MBX-4\MBX-4-SG1-Logs\
Env Log Sec size: 512
Env (CircLog,Session,Opentbl,VerPage,Cursors,LogBufs,LogFile,Buffers)
( off, 552, 27600, 15960, 27600, 2048, 2048,2000000000)
Using Reserved Log File: false
Circular Logging Flag (current file): off
Circular Logging Flag (past files): off

      Last Lgpos: (0xb8f,8,30)

Integrity check passed for log file: e00.log

Operation completed successfully in 0.63 seconds.

The signature is what we can use to make the most direct comparison.  Based on this information, we can confirm that the log file that was copied and placed in the SCR TARGET log directory matches the log from the SCR SOURCE log directory.  The command was actually successful.

As I indicated above we actually will attempt to copy the log file 6 times (max retires).  If you look in the management shell text, you will see that the log was moved to “f:\MBX-4\MBX-4-SG1-Logs\IgnoredLogs\E00OutofDate\2009-02-17T14-42-53E00.log” –> in this case the E00OutofDate directory inside ignored logs. 

 

image

 

If you review the folder you will see that the time stamps of the logs are all the same and correspond to the attempt to run restore-storagegroupcopy –standbymachine.  If you run eseutil /ml against one of the log headers, you can again compare the signatures and verify these logs both match the log successfully copied to the SCR TARGET directory and the log on the SCR SOURCE directory.

Extensible Storage Engine Utilities for Microsoft(R) Exchange Server

Version 08.01

Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...

      Base name: 200
Log file: 2009-02-17T14-41-58E00.log
lGeneration: 2959 (0xB8F)
Checkpoint: NOT AVAILABLE
creation time: 02/17/2009 09:26:41
prev gen time: 02/17/2009 09:26:40
Format LGVersion: (7.3704.12)
Engine LGVersion: (7.3704.12)
Signature: Create time:01/08/2009 10:39:04 Rand:47957569 Computer:
Env SystemPath: e:\MBX-4\MBX-4-SG1-System\
Env LogFilePath: f:\MBX-4\MBX-4-SG1-Logs\
Env Log Sec size: 512
Env (CircLog,Session,Opentbl,VerPage,Cursors,LogBufs,LogFile,Buffers)
( off, 552, 27600, 15960, 27600, 2048, 2048,2000000000)
Using Reserved Log File: false
Circular Logging Flag (current file): off
Circular Logging Flag (past files): off

      Last Lgpos: (0xb8f,8,30)

Integrity check passed for log file: e00.log

Operation completed successfully in 0.63 seconds.

Because of this issue repeat attempts to run restore-storagegroupcopy –standbymachine will return the same error.  There are a few things that can be done to correct this issue:

  • Use the information in this blog to manually verify log copy was successful.  If successful, run restore-storagegroupcopy –standbymachine <name> –force.
  • Contact Microsoft Customer Support Services.  There are incremental updates for both Exchange 2007 SP1 RU5 and Exchange 2007 SP1 RU6 currently available.

This issue is corrected in Exchange 2007 SP1 RU7.

This issue only affects Exchange 2007 SP1 Standby Continuous Replication when using any potential source.

(Note:  If you elect to receive an interim update, the IU only needs to be applied on the SCR target machine.)