Recover soft-deleted mailboxes in an Exchange Hybrid scenario

I have encountered a lot of situations where IT Administrators have difficulties in recovering a mailbox that was deleted from Office 365 active mailboxes, while having an Exchange Hybrid environment, so I’m creating this article to help Administrators perform a correct recovery and to avoid the situations like creating duplicate users with blank mailboxes.

Scenario1: Recover a mailbox that was deleted due to Directory Synchronization filtering changes:

Most of the organizations do not sync the whole Active Directory database and prefer to use filtering to narrow the Directory Synchronization (AADConnect) scope only to the users the company needs to sync to O365.

 

If a user is moved from a synced Organizational Unit (OU) into a non-synced OU, next time the AADConnect will perform a delta sync, the Office365 user will be moved from Active Users into Deleted Users and as a consequence the associated mailbox will be moved to Soft-Deleted mailboxes and kept there for 30days, during this time the administrator can very easily recover the deleted mailbox by applying the below logic:

Demo: For the purpose of this demo, I will use an AD account that has the following UserPrincipalName(UPN): PodByteSize1@scdtech.co

  1. Let’s reproduce the issue:
  • Log into the on-premises server that is hosting the Directory Synchronization engine (in my case AADConnect) and check what OUs are synced
    • For example, in my test environment I’m only synching one Organizational Unit (OU):

  • Move the user to a non-synced OU
  • Perform a sync:
    • Open a standard Windows Powershell window (on the server hosting the AADConnect) and run the below cmdlets:
      • Import-Module “C:\Program Files\Microsoft Azure AD Sync\Bin\ADSync\ADSync.psd1”
      • Start-ADSyncSyncCycle -PolicyType Delta
  • Check the AADSync client to spot the changes: You will notice that the AADConnect engine will export to Office365 a Delete event for this user.
  • Verify that the user was removed from O365:
    • Get-MsolUser –UserPrincipalName PodByteSize1@scdtech.co
    • Get-MsolUser –UserPrincipalName PodByteSize1@scdtech.co –ReturnDeletedUsers
  • Verify the Mailbox status:

    • Get-Mailbox PodByteSize1 | FT Identity, WhenCreated, WhenChanged
    • Get-Mailbox PodByteSize1 -SoftDeletedMailbox | FT Identity, WhenCreated, WhenChanged
    • Get-Mailbox PodByteSize1 -InactiveMailbox | FT Identity, WhenCreated, WhenChanged

Only the Get-Mailbox PodByteSize1 -SoftDeletedMailbox & Get-MsolUser –UserPrincipalName PodByteSize1@scdtech.co –ReturnDeletedUsers will return a result which is telling us that the mailbox and the user associated with it, are in a soft delete state.

What if the two cmdlets return multiple entries, for example what if the Get-MsolUser –UserPrincipalName <UPN> –ReturnDeletedUsers return two or more entries, which user is associated to our mailbox?

In this scenario, we do not really need to know the answer to the above question because the recovery is triggered by actions performed in the on-premises environment and the mailbox restauration is pretty straight forward and easy.

All you need to do is move the user back into the OU the user originally resided. Assuming that the OU the user was previously in is still being synchronized, the next time Directory Synchronization complete, the user and all associated data will be restored. By default, directory synchronizations occur every thirty minutes and after you move the user back to the proper OU you must wait for the next sync cycle to take place. However, if you are like me and cannot wait, then you can force the synchronization by running this cmdlet into Powershell: Start-ADSyncSyncCycle -PolicyType Delta

 

  • Move the user back to a synced OU
    • Open a PowerShell window and run Perform a sync: Start-ADSyncSyncCycle -PolicyType Delta
  • Check that the mailbox is in an active state: Get-Mailbox scenario1 | FT Identity, WhenCreated, WhenChanged -> You will notice that the mailbox is now active and it will be accessible by using the credentials of the user from AD.

Scenario2: Recover a mailbox when the on-premises Active Directory user is deleted (with recycle bin):

If your Active Directory is hosted on Windows Server 2008 R2 or above, then you can enable the Active Directory Recycle Bin feature. This feature is very useful in situations when you need to recover a user that was permanently deleted from you AD.

To verify if you AD has this feature enabled, you can check by running the following cmdlets in a Windows PowerShell on a Domain Controller:

  • Import-Module ActiveDirectory
  • Get-ADOptionalFeature -Filter 'name -like "Recycle Bin Feature"'

In the output, we have a parameter called EnabledScopes, if this parameter is blank then this feature is not enabled. When this feature is enabled, the value of EnabledScopes wil be similar to this one:

{CN=Partitions,CN=Configuration,DC=scdtech,DC=co, CN=NTDS

Settings,CN=ADVM,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=scdtech,DC=co}

This scenario addresses the situation when the Recycle Bin is enabled on the Active Directory.

Demo: For the purpose of this demo, I will use an AD account that has the following UserPrincipalName(UPN): PodByteSize2@scdtech.co

  • Go to a synced OU and delete a user (i.e. PodByteSize2@scdtech.co -> SyncedUsers OU)
  • Perform a sync.
  • Verify that the user was removed from O365.
    • Get-MsolUser –UserPrincipalName PodByteSize2@scdtech.co
    • Get-MsolUser –UserPrincipalName PodByteSize2@scdtech.co –ReturnDeletedUsers

Only the second cmdlet should return an entry.

  • Verify the Mailbox status:
    • Get-Mailbox PodByteSize2 | FT Identity, WhenCreated, WhenChanged
    • Get-Mailbox PodByteSize2 -SoftDeletedMailbox | FT Identity, WhenCreated, WhenChanged
    • Get-Mailbox PodByteSize2 -InactiveMailbox | FT Identity, WhenCreated, WhenChanged

Only the Get-Mailbox PodByteSize2 -SoftDeletedMailbox and Get-MsolUser –UserPrincipalName PodByteSize2@scdtech.co –ReturnDeletedUsers

should return an entry.

 

Now that we reproduced the scenario how do we recover this mailbox?

Solution:

  • First thing we need to do is to recover the AD user from recycle bin:

The below cmdlet will produce a list of all AD objects found in the dumpster and we need to check the properties to understand which user to restore, like the whenChanged date. If by analyzing this date, we it is still unclear which user we need to recover then we need to use PowerShell to see which ObjectGuid of the AD User matches the ImmutableID of the MsolUser(office 365):

  • On the Domain Controller in PowerShell run the following cmdlet:
    • [System.Convert]::ToBase64String((new-Object system.Guid("XXXXXXXXXXXXXXXXXXXXXXXX")).ToByteArray())

We need to replace the XXX… string with one of the ObjectGuids from the Get-AdObject cmdlet:

get-aduser1 converttobase641

As you can see from the above screenshot the cmdlet will transform the ObjectGuid into Base64 value, the ImmutableID.

  • Next step is to verify if this ImmutableID is an exact match of the property of the user in Office365:

getimmutableid

As you can see the two values are identical so now we know which user we need to recover from Recycle Bin.

  • To restore the AD user from the Dumpster to an active state we need to run the following PowerShell cmdlet:
    • Restore-ADObject -Identity <ObjectGuid>
  • Now that we have the user in an active state we just need to perform a Directory Synchronization and the user will be matching the deleted user in Office 365.
  • After the sync is done verify if the Cloud mailbox and user are active or not:
    • Get-MsolUser –UserPrincipalName PodByteSize2@scdtech.co
    • Get-Mailbox PodByteSize2 | FT Identity, WhenCreated, WhenChanged

The above cmdlets should return the value of the mailbox we have been trying to recover.

Scenario3: Recover a mailbox when the on-premises Active Directory user is deleted (with no recycle bin)

This scenario is coming into play when the AD user is deleted and the AD doesn’t have the Recycle Bin enabled, so we have no possibility to recover the AD user anymore.

Demo:

  • Go to a synced OU and delete a user (i.e. PodByteSize3@scdtech.co -> SyncedUsers OU)
  • Perform a sync.
  • Verify that the user was removed from O365.
    • Get-MsolUser –UserPrincipalName PodByteSize3@scdtech.co
    • Get-MsolUser –UserPrincipalName PodByteSize3@scdtech.co –ReturnDeletedUsers
  • Verify the Mailbox status:
    • Get-Mailbox PodByteSize3 | FT Identity, WhenCreated, WhenChanged, ExternalDirectoryObjectId
    • Get-Mailbox PodByteSize3 -SoftDeletedMailbox | FT Identity, WhenCreated, WhenChanged, ExternalDirectoryObjectId
    • Get-Mailbox PodByteSize3 -InactiveMailbox | FT Identity, WhenCreated, WhenChanged, ExternalDirectoryObjectId

Solution:

  • First step is to recover the Office365 MSOL user. We can achieve this by using remote PowerShell connected to Exchange Online (EXO) and to Azure Active Directory (MSODS):

restore-msol

  • Second step we need to do is to create an on-premises remote mailbox and then match the on-premises identity with the cloud one:
  • Restore-MsolUser -ObjectId <ObjectID from the deleted user>

new-remotemailbox

  • New-RemoteMailbox -Name "mailbox name" -UserPrincipalName <UPN> -RemoteRoutingaddress <Alia>@domain.mail.onmicrosoft.com -OnPremisesOrganizationalUnit “OU path“

immutableid-check

  • Prepare for a “hard match” by using the login from the above screenshot.

set-msol

  • Stamp the MsolUser with the ImmutableId resulted from the [System.Convert]::ToBase64String((new-Object system.Guid("XXXXXXXXXXXXXXXXXXXXXXXX")).ToByteArray()) cmdlet
  • Last step is to perform a delta sync and the on-premises user will be in-sync with the cloud user/mailbox.