Using the mS-DS-ConsistencyGuid attribute to fix sync issues to Office 365/AAD.


By Cesar Hara, Victor Santana and Caio Cesar

Greetings everyone!

Today we are going to cover a very interesting way to troubleshoot user synchronization issues to Office 365 (Azure Active Directory). The method consists of using the attribute: “mS-DS-ConsistencyGuid”.

We recommend going through this article for a better understanding of what is being discussed in this post.

As many of you know, historically, AD Connect used to choose “ObjectGUID” as the sourceAnchor attribute to sync objects to the cloud. Starting on version 1.1.524.0, we can use mS-DS-ConsistencyGuid instead.

What is the big deal with it? Unlike ObjectGuid, this attribute is writable. This means we can manipulate synced objects easier.

Recently, customers started to raise tickets related to this matter. We were able to fix these incidents without causing service disruption to the end users, scenarios may vary from: “I have a synced multi-forest environment and the user was migrated between them”, “A user was deleted from Active Directory by mistake while AD Connect was not working and we reinstalled it” and “I lost the only DC in my forest, so I had to rebuild my AD forest completely”.
(PLEASE, ALWAYS PROMOTE AT LEAST TWO DOMAIN CONTROLLERS!!)

Pre-requisites: You will need to have access to AAD PowerShell connected to your tenant (with global admin permissions), AD Connect Server and access to the Domain Controller.

Scenario 1:

“A user was deleted from AD by mistake while AD Connect was not working. We created a new account since we cannot recover the old one and then we reinstalled AD Connect, but the changes performed on local Active Directory object are not syncing to the cloud account.”
NOTE: There are other possible scenarios, this is only one of the scenarios we have seen.

Understanding the problem:

• As AD Connect was down, the removal of the user was not “picked” by AD Connect.
• As a result, this deletion did not occur in AAD. This means the account is orphan in AAD.
• The new account has a new on-prem ImmutableID (originated from ObjectGUID). This attribute cannot be changed in the cloud for synced accounts.
To reproduce this behavior, we have uninstalled AD Connect, deleted the user account from Active Directory and re-installed AD Connect.

To confirm that the problem is with the different ImmutableID, you can follow these steps:
• Get the ImmutableID of the affected account in the cloud:

Get-MsolUser -UserPrincipalName john@cesarhara.com | select ImmutableID
ImmutableID: kKfL2wwI+0W+rN0kfeaboA==

• Perform a metaverse search for the new user created in AD (or convert the ObjectGUID taken from AD into a base64 format with the GUID2ImmutableID tool) to confirm the new ImmutableID:

• If you have attribute resiliency, AD Connect will not show any errors. You can go to the Portal or PowerShell to identify the error:

In summary, any changes we make to this on-prem object will not be reflected to the cloud.

Steps to resolve:

1- Confirm that the attribute being used in AD Connect as the “SourceAnchor” is the “mS-DS-ConsistencyGuid”, just run the AD Connect Wizard and choose “View Current Configuration”:

2- Make sure that the “UPN” and “ProxyAddresses” attributes are the same in O365 and AD accounts - you can download the users report in the Admin Portal or use PowerShell to extract this information. These attributes are essential, if you are aware of any other one missing, proceed to add it to the account in your local AD accordingly.

3- Grab the ImmutableID value of the cloud/original account. Here is the cmdlet again:

Get-MsolUser -UserPrincipalName john@cesarhara.com | select ImmutableID
ImmutableID: kKfL2wwI+0W+rN0kfeaboA==

4- We have to convert the value of the “ImmutableID” to a HEX format so we can add it into the on-prem account:

[system.convert]::FromBase64String("kKfL2wwI+0W+rN0kfeaboA==") | %{$a += [System.String]::Format("{0:X}", $_) + " "};$result = $null;$result = $a.trimend();$result

The output is:
90 A7 CB DB C 8 FB 45 BE AC DD 24 7D E6 9B A0

5- Now let’s get back to our AD Connect server and stop the scheduled task (it’s a good practice to do so, avoiding a sync task to run while we are troubleshooting this):

Set-ADSyncScheduler -SyncCycleEnabled $false

6- Move the account to a non-synced OU and run a delta sync (this is required to the match occurs later).

Start-ADSyncSyncCycle -PolicyType Delta

7- In the new account created in ADDS, edit its attributes. Do this by locating the mS-DS-ConsistencyGuid and adding the HEX value from step 2 (if there is a value, AD Connect added the current ObjectGUID value as the default behavior, proceed to add the new one):

8- Move the account back to a synced OU and run a delta sync again, if everything ran as expected the sync would occur successfully.

9- Add or change an attribute in AD to make sure the sync worked as expected. Check in the Portal and the error will be gone:

10- Finally, let’s enable the sync scheduler again:

Set-ADSyncScheduler -SyncCycleEnabled $True

 

Conclusion:

With this new feature, it is possible to resolve problems that used to be hard to deal with it when using “ObjectGUID”. Regardless the scenario, we must always use the original ImmutableID (already set in the cloud), convert it to an HEX value and add into the “mS-DS-ConsistencyGuid” so the match occurs.
If you are going to migrate accounts between forests make sure to populate in the target forest object “mS-DS-ConsistencyGuid”, the value of the source object “ObjectGUID”.


Comments (5)

  1. Jay Antoney says:

    ** Don’t forget to re-enable the sync

    Set-ADSyncScheduler -SyncCycleEnabled $true

    1. Cesar Hara says:

      Thanks for the tip, I have updated the content.

  2. My situation is a bit in the twilight zone, but at last after weeks of searching, I found your article, which addresses the problem I am having. 100 users with Office 365 subscriptions through GoDaddy. Thought it would be neat to have password hash sync at the least, so on-premises AD user accounts to sync/connect with GoDaddy/Azure accounts. MS tech support told me this was possible, and have had an Azure support ticket open with them for weeks. GoDaddy advised me after going down this path that this is not possible. So, all existing user accounts in Azure already had ImmutableIDs set, which is what federates them to GoDaddy. MS tech support had me sync all users, and all became conflicting objects. We then nulled out the ImmutableID on Azure of one poor user, which then allowed that user to sync, but broke his ability to log in to the GoDaddy/Azure partner portal and could not check his email of course, because couldn’t authenticate. Error was user ID xxxxx not found in directory xxxxx. Solution was to have GoDaddy level 2 support reset the user account on their side to the new ImmutableID that had be generated from AAD Sync. At this point, I had this user in an OU called TEST by himself. Everything was working again, but then a week later I decided to turn off AAD Sync by moving his AD account back into the OU with all the other users onsite, and then going into the Syncr. Service Manager and unselecting the TEST OU as a sync Container, so nothing was selected to sync. The effect of this was the next morning the user’s account had be deleted out of Azure. I found it and restored the account, but now the ability to sign-in from the GoDaddy/Office 365 portal and get his email was broken again. GoDaddy level 2 support again had to reset the ImmutableID on their side to match what was on the Azure side. This time, they took three days to do it, because I think they were mad. In the meantime, I had called MS support back, and their solution was to re-enable sync for everyone, which created 99 conflicting objects again, with the one user that we had tossed around for a couple of weeks being the only one that works correctly federated to GoDaddy and sync with onsite AAD.
    So, I can see and have tested my ability to see the two conflicting ImmutableIDs, and know that the current Azure side one must not be changed, and must be converted to HEX and updated to the onsite AD account properties. I have done the HEX conversion for one account with sync error status and have the HEX number ready. My concern and question is that if I turn off sync, move this user out of the SyncUsers OU, update his ImmutableID with the one from Azure, that something bad will happen, like his Azure account with his active subscriptions will be deleted from Azure during the process, and I would have to restore it and then it might not work correctly and I would have to bother GoDaddy tech support again with my foolishness. How do I update the ImmutableID of an onsite AD account so that the next time a AAD Sync runs, it removes the error status object from Azure and does a proper Hard Match to the existing Azure account? Thank you for reading all this if you got this far!

    1. Cesar Hara says:

      Hello, sorry the late reply. Were you able to resolve that? I am not aware how go daddy works with the integration with AAD/O365, but this seems a bit challenging for your needs.

Skip to main content