4133 Database one datacenter health check failed. 4376 Database one datacenter available copy health check failed.
“Every day brings a new challenge…”
I am still convinced, that whoever said that first, was an Exchange Server Administrator…
If you wake up one day and find these two beautiful events in your Application Logs, no need to wonder:
4133 Database one datacenter health check failed.
4376 Database one datacenter available copy health check failed.
Log Name: Application
Source: MSExchangeRepl
Date: …
Event ID: 4133
Task Category: Service
Level: Error
Keywords: Classic
User: N/A
Computer: …
Description:
Database one datacenter health check failed.
Database copy: …
Redundancy count: 2
Error:
================
Full Copy Status
================
…
Log Name: Application
Source: MSExchangeRepl
Date: …
Event ID: 4376
Task Category: Service
Level: Error
Keywords: Classic
User: N/A
Computer: …
Description:
Database one datacenter available copy health check failed.
Database copy: …
Redundancy count: 2
Error:
================
Full Copy Status
================
…
After performing some tests in our Exchange 2013 test environments, we managed to find out that these events could be expected in the following scenario:
spanned DAG over multiple physical locations but in one AD site, containing at least 2 nodes
created Passive copies for the DBs
enabled DAC Mode for DAG (DagOnly)
Results:
These errors were thrown only after enabling DAC Mode on the DAG (errors started to be thrown for all DBs and all their copies that existed on nodes in the same AD site)
Please note that these errors can be thrown also in a scenario where DAG members are split over 2 or more AD sites, but where Databases are replicated only over nodes in the same AD site.
What to do?
Not to panic! These events can be safely ignored. They do not refer to any issues with the DAC mode, nor with replication between nodes.
*** Please make sure however that you are not hitting another issue. You must check the “Error” Section in the description of these events (both Event ID 4133 and 4376). In our case described in this blog, you will see this Error Section is empty:
If this section is not empty, but populated with extra information, as in the screenshot below, please troubleshoot it separately, as this is not our scenario:
The information in this Error section might indicate real issues which need further investigation. So please do not ignore the events, if this error section is populated.
You can disable the DAC mode (these errors will not be logged anymore, if DAC mode disabled)
You can span the DAG members on two AD Sites and make sure each DB has a copy in each site, the errors will not be logged anymore.
You can wait for CU7, as there are plans to change the design in a way that will not log these errors in the Application Logs any more but in the „Monitoring“ Crimson Channel Log and that will add relevant text in the Error section in these events, so that you can have a better understanding of their cause.