A customer reported an issue that they could not protect virtual machines running various Microsoft workloads on a Windows Server 2008 R2 Enterprise Hyper-V Server. In Hyper-V terms, these virtual machines are defined as “Child Partitions”. More information about architecture can be found here:
At a high level, "The Volume Shadow Copy Service (VSS) is a set of COM interfaces that implements a framework to allow volume backups to be performed while applications on a system continue to write to the volumes. Using a combination of flushing changes to disk and holding writes allows the system to become in a consistent state for backup."
Overview of Processing a Backup Under VSS
In my customer’s case, they are using (DPM) Microsoft Data Protection Manager 2010. The error message reported in the jobs log was a consistency check failure.
Type: Consistency check
Description: The VSS application writer or the VSS provider is in a bad state. Either it was already in a bad state or it entered a bad state during the current operation. (ID 30111 Details: VssError:The writer experienced a non-transient error. If the backup process is retried,
the error is likely to reoccur.
End time: 2/2/2011 5:25:54 PM
Start time: 2/2/2011 5:15:17 PM
Time elapsed: 00:10:37
Data transferred: 0 MB
Cluster node -
Source details: \Backup Using Child Partition Snapshot\MYBLOGVM
Protection group: BLOGGroup
Items scanned: 0
Items fixed: 0
# for hex 0x800423f4 / decimal -2147212300 :
# indicates that the writer failed due to an error
# that most likely would occur if another snapshot is created
# 1 matches found for "0x800423F4"
Based on this error we can conclude that this is not necessarily a DPM issue but a VSS issue preventing DPM from working properly. The first thing I always recommend is ensuring that DPM is running with the latest rollup available. At the time this blog was authored, I applied the following update.
System Center Data Protection Manager 2010 QFE Rollup [KB 2250444]
After installing this update, the job still failed. The error within DPM had a link called “More Information” referencing the following information:
Automount should be enabled on all nodes
Error: (ID 30111 Details: Unknown error (0x800423f4) (0x800423F4))
Posible solution: Enable automount (cmd /c mountvol.exe /E) on the Hyper-V host (on all nodes in a cluster if the hosts are part of a Failover Cluster)
This Hyper-V server is not a part of a cluster but the error code is specific so I checked whether the automount feature of Windows was enabled, which is the default. The system had automount disabled and after some research found that this is a requirement for child partitions to be backed up via VSS Hyper-V Writer. I enabled automount and restarted consistency check jobs successfully!
Dave Guenthner [MSFT]
This posting is provided "AS IS" with no warranties, and confers no rights.