Migrating Hybrid Public Folders to Office 365

So, tonight I started the last phase of one of my longest-running projects since joining Microsoft--an Exchange Online migration for a school district that I began nearly a year and a half ago.  40,000 mailboxes down and 13,000 public folders remaining.

One of the things that we recommend for Hybrid Public Folders is that you run the SyncMailPublicFolders.ps1 script daily.  This can be a pain (as someone has to remember do it), and because it requires an Office 365 credential, you have two options: modify and hard code it in the script or store it in a more secure location, such as a service account's local registry.

// I will digress here momentarily

While we don't advocate storing passwords in plain-text in a script, I will show you how it's done for this script for purposes of education.  I also have code here that will show you how to do the same procedure, only encoding the data in the registry under the service account's context (which is what we did in this case).

To make this change for the purpose of testing automation:

  • Comment out the Credential parameter (lines 47 and 48)
  • Set the CsvSummaryFile output (line 52) to a file that will just get overwritten every day.
  • Add a code block for the credential (insert the following after the param() block and before the WriteInfoMessage() function)
# Password/credential
$userAdmin = "tenantadmin@tenant.onmicrosoft.com"
$userAdminPass = "Password123"
$securePassword = ConvertTo-SecureString $userAdminPass -AsPlainText -Force
$global:Credential = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist $userAdmin,$securePassword

It will look something like this:

This will now allow the SyncMailPublicFolders.ps1 script to run as a scheduled task.  Please, for the love all that is holy, don't do this in production--use a more secure method to store credentials in an encrypted fashion.

Now, back to our regularly scheduled post.



  • Planning for the time to do an event like this is really a crap shoot.  This customer moved 40,000 mailboxes with 2TB of data over the course of about 40 days.  This public folder data migration had 131GB of data and took 6 full days.  Since legacy public folders are migrated into Public Folder Mailboxes (modern public folders), they behave more like mailbox migrations and can be simultaneous.  Your biggest bottlenecks will be read I/O performance on your source environment and the bandwidth + latency to get to an Office 365 datacenter.
  • Every time I tell you to run something from your Exchange On-Premises organization, I mean "run this from the Exchange Management Shell in on your Public Folder server, unless otherwise specified."
  • Every time I tell you to do something in your Exchange Online or Office 365 tenant, I mean "run this from a PowerShell session connected to Office 365 FROM YOUR EXCHANGE ON-PREMISES SERVER."  You can do it otherwise, but this way will save you a lot of copying and pasting and frustration.
    • If your on-premises organization is 2007, you'll need to just connect to the Exchange Online endpoint (you won't need to nor will you be able to install the Windows Azure AD PowerShell for the MSOL cmdlets, so use this three-liner to get connected to Exchange Online directly:
      $o365Cred = Get-Credential
      $o365Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid -Authentication Basic -AllowRedirection -Credential $o365Cred
      Import-PSSession $o365Session

Preparing the Environment

  1. When I set up my customer for Hybrid Public Folders, I use a script that I put together to identify public folders with invalid characters.  This has helped many customers avoid nasty scenarios.   I stole the idea for the name (IDFix) from our tool of a similar name. You should run it in your environment to see what you get back. You can get it here: IDFix for Public Folders.
  2. Before I even began the night's activities, I needed to make sure the SyncMailPublicFolders.ps1 script I had automated was stopped.  No sense ruining things before I even start.
  3. The first thing I had to do is make sure that my customer had no existing Public Folder migrations.  Just because I didn't start one doesn't mean they didn't.
    Run from both Exchange Online and your Exchange On-Premises Servers
    Get-OrganizationConfig | FL Public*Migration*
    PublicFoldersLockedForMigration        : False
    PublicFolderMigrationComplete          : False
    PublicFolderMailboxesMigrationComplete : False
  4. If you have True reported for any of those, STOP.  Do not pass GO, do not collect $200.  You'll have to do some cleanup first.
    Run this from your Exchange On-Premises Org.

    Set-OrganizationConfig -PublicFoldersLockedforMigration:$false -PublicFolderMigrationComplete:$false
  5. Run this from your Office 365 tenant to make sure no one has tried monkeying around with public folder migrations behind your back.
    Get-PublicFolderMigrationRequest | Remove-PublicFolderMigrationRequest -Confirm:$False
    $PFMigrationBatch = Get-MigrationBatch | ? { $_.MigrationType.ToString() -eq "Public Folder" }
    $PFMigrationBatch | Remove-MigrationBatch -Confirm:$False
    Get-Mailbox -PublicFolder
    Get-MailPublicFolder | where {$_.EntryId -ne $null}| Disable-MailPublicFolder -Confirm:$false 
    Get-PublicFolder -GetChildren \ | Remove-PublicFolder -Recurse -Confirm:$false
    $hierarchyMailboxGuid = $(Get-OrganizationConfig).RootPublicFolderMailbox.HierarchyMailboxGuid
    Get-Mailbox -PublicFolder:$true | Where-Object {$_.ExchangeGuid -ne $hierarchyMailboxGuid} | Remove-Mailbox -PublicFolder -Confirm:$false
    Get-Mailbox -PublicFolder:$true | Where-Object {$_.ExchangeGuid -eq $hierarchyMailboxGuid} | Remove-Mailbox -PublicFolder -Confirm:$false
  6. Now that we're moderately confident that we can move forward, we need to make sure that we don't exceed the folder quota for any posts that might be migrating in.
    Get-OrganizationConfig | fl *quot*
    DefaultPublicFolderIssueWarningQuota : 1.7 GB (1,825,361,920 bytes)
    DefaultPublicFolderProhibitPostQuota : 2 GB (2,147,483,648 bytes)

    No, that definitely won't do.

  7. So, we update the Public Folder quotas.
    Set-OrganizationConfig -DefaultPublicFolderIssueWarningQuota 9.5GB -DefaultPublicFolderProhibitPostQuota 10GB
  8. Next, it's time to gather some data before we begin.  Since my customer's source environment was Exchange 2007, I needed to log into one of those servers and launch the Exchange Management Shell there.  I like to do belt and suspenders, so I ran some extra data gathering commands.  You can run this if you like.  Your mileage may vary, but I've found it helpful in the past to have backups of your backups.
    Get-PublicFolder -Recurse -ResultSize Unlimited | Export-Clixml .\LegacyPFStructure.xml
    Get-PublicFolder -Recurse -ResultSize Unlimited | Get-PublicFolderStatistics | Export-Clixml .\LegacyPFStatisticsRecurse.xml
    Get-PublicFolder -Recurse -ResultSize Unlimited | Get-PublicFolderClientPermission | Select-Object Identity,User -ExpandProperty AccessRights | Export-Clixml .\LegacyPFPerms.xml
  9. But, I also grabbed our Public Folder Migration Scripts and saved them to my customer's Exchange 2007 server in C:\PFScripts (just like the document says, because I'm lazy).
  10. The next step is an internal relay domain with a well-known name that we can use to route mail during the transition.  Run this from your Exchange On-Premises Org.
    New-AcceptedDomain -Name "PublicFolderDestination_78c0b207_5ad2_4fee_8cb9_f373175b3f99" -DomainName tenant.onmicrosoft.com -DomainType InternalRelay
  11.  Just in case we missed anything in the Public Folder names, we're going to check and make sure.  If you're still unlucky enough to be running Exchange 2007 like this customer is, this is the command that you'll run.  Run this from your Exchange On-Premises Org.

    Get-PublicFolderDatabase | ForEach {Get-PublicFolderStatistics -Server $_.Server | Where {$_.Name -like "*\*"}}

    If you're in the modern era and have upgraded to Exchange 2010, you can use this command:

    Get-PublicFolderStatistics -ResultSize Unlimited | Where {($_.Name -like "*\*") -or ($_.Name -like "*/*") } | Format-List Name, Identity
  12. One of the meh things about Public Folder migrations is that Send-As and Send-On-Behalf permissions aren't migrated.  So, we're going to grab them as well so we can apply them later.  Run this from your Exchange On-Premises Org.Send-As:
    Get-MailPublicFolder -ResultSize Unlimited | Get-ADPermission | ? {($_.ExtendedRights -Like "Send-As") -and ($_.IsInherited -eq $False) -and -not ($_.User -like "*S-1-5-21-*")} | Select Identity,User | Export-Csv Send_As.csv -NoTypeInformation


    Get-MailPublicFolder | Select Alias,PrimarySmtpAddress,@{N="GrantSendOnBehalfTo";E={$_.GrantSendOnBehalfTo -join "|"}} | Export-Csv GrantSendOnBehalfTo.csv -NoTypeInformation
    $File = Import-Csv .\GrantSendOnBehalfTo.csv
    $Data = @()
    Foreach ($line in $File)
        If ($line.GrantSendOnBehalfTo)
            Write-Host -ForegroundColor Green "Processing Public Folder $($line.Alias)"
            [array]$LineRecipients = $line.GrantSendOnBehalfTo.Split("|")
            Foreach ($Recipient in $LineRecipients)
                Write-Host -ForegroundColor DarkGreen "     $($Recipient)"
                $GrantSendOnBehalfTo = (Get-Recipient $Recipient).PrimarySmtpAddress
                $LineData = New-Object PSCustomObject
                $LineData | Add-Member -Type NoteProperty -Name Alias -Value $line.Alias
                $LineData | Add-Member -Type NoteProperty -Name PrimarySmtpAddress -Value $line.PrimarySmtpAddress
                $LineData | Add-Member -Type NoteProperty -Name GrantSendOnBehalfTo -Value $GrantSendOnBehalfTo
                $Data += $LineData
    $Data | Export-Csv .\GrantSendOnBehalfTo-Resolved.csv -NoTypeInformation
  13. Now, for the funsies.  I'm going to make a new sub-directory to store my working files in.  Again, for reference, I've saved all of the Microsoft-provided Public Folder scripts in C:\PFScripts.
    Run this on your Exchange On-Premises Org.

    cd \PFScripts
    Mkdir C:\PFScripts\PFMigrate
    .\Export-PublicFolderStatistics.ps1 C:\PfScripts\PFMigrate\PFStatistics.csv publicfolderserver.domain.com
  14.  While processing, I saw this interesting bit of information:
    [4/8/2017 3:31:30 AM] Enumerating folders under IPM_SUBTREE...
    [4/8/2017 3:42:04 AM] Enumerating folders under IPM_SUBTREE completed...13039 folders found.
    [4/8/2017 3:42:04 AM] Enumerating folders under NON_IPM_SUBTREE...
    [4/8/2017 3:42:06 AM] Enumerating folders under NON_IPM_SUBTREE completed...118 folders found.
    [4/8/2017 3:42:06 AM] Retrieving statistics from server PUBFOLD01
    [4/8/2017 3:42:12 AM] Retrieving statistics from server PUBFOLD01 complete...13064 folders found.
    [4/8/2017 3:42:13 AM] Total unique folders found : 13064.
    [4/8/2017 3:42:13 AM] Retrieving statistics from server PUBFOLD02
    [4/8/2017 3:42:20 AM] Retrieving statistics from server PUBFOLD02 complete...13050 folders found.
    [4/8/2017 3:42:21 AM] Total unique folders found : 13072.
    [4/8/2017 3:42:21 AM] Exporting statistics for 13072 folders
    [4/8/2017 3:42:23 AM] Folders processed : 10000.
    [4/8/2017 3:42:24 AM] Exporting folders to a CSV file

    What am I supposed to do with this information?  The script clearly says that it found 13,072 unique folders, but this screen output makes it seem like it's only processing 10,000.  And not like it ran into a problem 10,000.  This seems like a value that someone thought would be enough.  That doesn't really sound like it's going to work for my customer.

    "Hey, I saw you had 13,072 public folders, but I only decided to grab 10,000 of them.  Good luck!"

    Just let me know if you want any other advice for helping you free up your future to plan for another career.

    So, I did what any consultant would do, and decided to tinker.  I opened Export-PublicFolderStatistics.ps1 and searched for 10000.  Found it on line 154 inside of the CreateFolderObjects() function. Made a copy of the line, commented it out, and updated the number of 100000 and saved.


  15. Reran the command and received what I deemed to be better output.
    [4/8/2017 3:47:27 AM] Enumerating folders under IPM_SUBTREE...
    [4/8/2017 3:57:53 AM] Enumerating folders under IPM_SUBTREE completed...13039 folders found.
    [4/8/2017 3:57:53 AM] Enumerating folders under NON_IPM_SUBTREE...
    [4/8/2017 3:57:55 AM] Enumerating folders under NON_IPM_SUBTREE completed...118 folders found.
    [4/8/2017 3:57:55 AM] Retrieving statistics from server PUBFOLD01
    [4/8/2017 3:53:02 AM] Retrieving statistics from server PUBFOLD01 complete...13064 folders found.
    [4/8/2017 3:53:02 AM] Total unique folders found : 13064.
    [4/8/2017 3:53:02 AM] Retrieving statistics from server PUBFOLD02
    [4/8/2017 3:53:09 AM] Retrieving statistics from server PUBFOLD02 complete...13050 folders found.
    [4/8/2017 3:53:10 AM] Total unique folders found : 13072.
    [4/8/2017 3:53:10 AM] Exporting statistics for 13072 folders
    [4/8/2017 3:53:13 AM] Exporting folders to a CSV file
  16. Now it's time to generate the Public Folder map file, which iterates through the CSV you just created and figures out how to distribute the public folders into Public Folder Mailboxes in Office 365.  In this case, I told the script to break the public folders at 10GB boundaries:
    .\PublicFolderToMailboxMapGenerator.ps1 10000000000 C:\pfscripts\PFMigrate\PFStatistics.csv C:\pfscripts\PFMigrate\PFMap.csv

    Data is returned that looks acceptable:

    [4/8/2017 4:02:22 AM] Reading public folder list...
    [4/8/2017 4:02:22 AM] Loading folder hierarchy...
    [4/8/2017 4:02:24 AM] Allocating folders to mailboxes...
    [4/8/2017 4:02:24 AM] Trying to accomodate folders with their parent...
    [4/8/2017 4:02:24 AM] Exporting folder mapping...

    In case you're like me and always curious, you can look inside the PFMap.csv file (or whatever you call it, but hopefully you're smart enough to figure that out without me explicitly pointing it out) to see what kind of data it generated:

    As you can see, it's got two columns--one named "TargetMailbox" (which will be the Public Folder Mailbox in Office 365 into which that particular branch of the public folder tree will be placed) and the "FolderPath" (which is, not surprisingly, the aforementioned particular branch).  Sorry if I spoiled the surprise.  You didn't come here for surprises.

Starting the Migration

  1. Switching over to the PowerShell session that is connected to Office 365, I run the following:
    cd \PFScripts
    .\Create-PublicFolderMailboxesForMigration.ps1 -FolderMappingCsv C:\pfscripts\PFMigrate\PFMap.csv -EstimatedNumberOfConcurrentUsers 40000

    I chose 40,000, since that's the number of users in the organization that will most likely be connecting to and browsing public folders.  The script returns some data:

    Public Folder mailbox updates.
    Creating 14 Public Folder mailbox(es) and updating 0. Total mailboxes to serve hierarchy will be 20. Would you like to proceed?
    [Y] Yes  [N] No  [?] Help (default is "Y"):

    Well, duh. Of course I want to continue. I didn't stay up until 4am to cancel a migration.

    [falls asleep]

    Ok, I'm back.  The recommended number of users per hierarchy-serving mailbox is 2,000.  Since I entered 40,000 users, the script has determined that I need at least 20 mailboxes to serve the public folder hierarchy.

    Carry on.

  2. After I hit Yes, the Create-PublicFolderMailboxesForMigration.ps1 script will go about its business of creating Public Folder mailboxes.  Run this from your Exchange Online Tenant.
    And now we wait.
  3. When it gets to the end, you can see that it created additional mailboxes:
    Onward and upward.
  4. Run the SyncMailPublicFolders.ps1 script one last time to make sure everything is working. If you already have hybrid public folders enabled, you know how to do this.  If you don't:
    .\SyncMailPublicFolders.ps1 -Credential (Get-Credential) -CSVSummarFile:csvoutput.csv-Credential is your Office 365 global admin.
  5. Now, we have some additional information to gather.  You can save this into variables.  Run this from your Exchange on-premises org in the Exchange Management Shell.
    (Get-Mailbox <admin user>).legacyExchangeDN | Out-File .\MailboxLegacyExchangeDN.txt
    (Get-ExchangeServer <public folder server>).ExchangeLegacyDN | Out-File .\ServerExchangeLegacyDN.txt
    $OAEndpoint = (Get-ExchangeServer)[0].ExternalHostNameHostnameString
    $OAEndpoint | Out-File .\OAEndpoint.txt

    That last one might be tricky if your org has multiple endpoints or if, in my customer's case, you never updated external Autodiscover to the hybrid endpoint per our recommendation.  What you're really after is the Autodiscover endpoint (2013 or later) that Office 365 will use to contact your on-premises environment.  In my customer's environment, it's hybrid.domain.edu, as is referenced in the OrgRelationship:

    Get-OrganizationRelationship | fl TargetAutoDiscoverEpr
    TargetAutodiscoverEpr : https://hybrid.domain.edu/autodiscover/autodiscover.svc/WSSecurity

    So, I need *that* value instead.

    $OAEndpoint = "hybrid.domain.edu"
    $OAEndPoint | Out-File .\OAEndPoint.txt
  6. Now, you need to switch back over to the PowerShell prompt that is connected to Exchange Online (hopefully you're running it from your PF server like I already told you to, or you're going to have some copying and pasting to do).
    cd \PFScripts
    $OAEndopint = gc .\OAEndpoint.txt
    $MailboxLegacyExchangeDN = gc .\MailboxLegacyExchangeDN.txt
    $ServerExchangeLegacyDN = gc .\ServerExchangeLegacyDN.txt
    $Credential = Get-Credential <domain\admin user>
  7. Now, it's time to create the Public Folder Migration Endpoint.  Run this in your Exchange Online Organization.
    $PFEndpoint = New-MigrationEndpoint -PublicFolder -Name PublicFolderEndPoint -RpcProxyServer $OAEndPoint -Credentials $Credential -SourceMailboxLegacyDN $MailboxLegacyExchangeDN -PublicFolderDatabaseServerLegacyDN $ServerExchangeLegacyDN -Authentication Basic
  8. Once the endpoint is created, I create the migration batch.
    New-MigrationBatch -Name PublicFolderMigration -CSVData (Get-Content .\PFMigrate\PFMap.csv -Encoding Byte) -SourceEndpoint $PFEndpoint.Identity -NotificationEmails <my email address>

    It processes for a minute, and then gives me some feedback.

    Identity                        Status                    Type                           TotalCount
    --------                        ------                    ----                           ----------
    PublicFolderMigration           Stopped                   PublicFolder                   14
  9. Time to get down to brass tacks.  I cross my fingers and start the migration batch.
    Start-MigrationBatch PublicFolderMigration

    And without any fanfare at all--no ribbons, no streamers, no Def Leppard songs over giant speakers--my migration begins.

  10. Before leaving my workstation to grab a Diet Coke (to celebrate my apparent victory over Public Folders), I decided run a quick check (Run this from your Exchange Online Org):
    Get-MigrationUser -BatchID PublicFolderMigration
    Identity                                 Batch                          Status                    LastSyncTime
    --------                                 -----                          ------                    ------------
    Mailbox1                                 PublicFolderMigration          Syncing                   4/8/2017 12:17:01 AM
    Mailbox2                                 PublicFolderMigration          Syncing                   4/8/2017 12:17:09 AM
    Mailbox3                                 PublicFolderMigration          Syncing                   4/8/2017 12:17:18 AM
    Mailbox4                                 PublicFolderMigration          Syncing                   4/8/2017 12:17:31 AM
    Mailbox5                                 PublicFolderMigration          Syncing                   4/8/2017 12:17:41 AM
    Mailbox6                                 PublicFolderMigration          Syncing
    Mailbox7                                 PublicFolderMigration          Validating
    Mailbox8                                 PublicFolderMigration          Validating
    Mailbox9                                 PublicFolderMigration          Validating
    Mailbox10                                PublicFolderMigration          Validating
    Mailbox11                                PublicFolderMigration          Validating
    Mailbox12                                PublicFolderMigration          Validating
    Mailbox13                                PublicFolderMigration          Validating
    Mailbox14                                PublicFolderMigration          Validating

    Sweet. Diet Coke it is.  I might even take some bourbon with it.

  11. Once all of the migration requests are underway, I tell the customer that Public Folders will be unavailable.  I lock the Public Folders so users can't make changes.  Run this in the Exchange On-Premises Org.

    Set-OrganizationConfig -PublicFoldersLockedForMigration:$true
  12. At this point, it's going to be watching some paint dry, but I like seeing pretty statistics, so I'll show you some more.  Run this from the Exchange Online Org.
    Get-MigrationUser -BatchId PublicFolderMigration | Get-MigrationUserStatistics | FT -auto
    Identity  Batch                 Status  Items Synced Items Skipped
    --------  -----                 ------  ------------ -------------
    Mailbox1  PublicFolderMigration Syncing 0            0
    Mailbox2  PublicFolderMigration Syncing 0            0
    Mailbox3  PublicFolderMigration Syncing 0            0
    Mailbox4  PublicFolderMigration Syncing 8279         0
    Mailbox5  PublicFolderMigration Syncing 0            0
    Mailbox6  PublicFolderMigration Syncing 0            0
    Mailbox7  PublicFolderMigration Syncing 0            0
    Mailbox8  PublicFolderMigration Syncing 0            0
    Mailbox9  PublicFolderMigration Syncing 2874         0
    Mailbox10 PublicFolderMigration Syncing 38535        0
    Mailbox11 PublicFolderMigration Syncing 0            0
    Mailbox12 PublicFolderMigration Syncing 3304         0
    Mailbox13 PublicFolderMigration Syncing 0            0
    Mailbox14 PublicFolderMigration Syncing 0            0

    There's actually a bit more useful data that can be gathered, so I grabbed a few of the extra fields from Get-MigrationUserStatistics:

    Get-MigrationUser -BatchId PublicFolderMigration | Get-MigrationUserStatistics | Select Identity,Status,SyncedItemCount,SkippedItemCount,BytesTransferred,PercentageComplete | Sort -prop Status,PercentageComplete | Ft -auto
    Identity  Status  SyncedItemCount SkippedItemCount BytesTransferred               PercentageComplete
    --------  ------  --------------- ---------------- ----------------               ------------------
    Mailbox1  Syncing           32553              176 2.092 GB (2,246,307,901 bytes)                 40
    Mailbox2  Syncing           95156                0 5.021 GB (5,391,041,974 bytes)                 94
    Mailbox3  Syncing           82711                0 2.987 GB (3,207,776,110 bytes)                 47
    Mailbox4  Syncing           23625                0 8.373 GB (8,990,905,121 bytes)                 20
    Mailbox5  Syncing            1584                0 1.99 GB (2,136,923,781 bytes)                  94
    Mailbox6  Syncing           51243                0 2.562 GB (2,750,709,675 bytes)                 70
    Mailbox7  Syncing           17659                0 3.26 GB (3,500,029,634 bytes)                  47
    Mailbox8  Syncing            6472                0 7.234 GB (7,767,318,075 bytes)                 94
    Mailbox10 Syncing          281718                1 3.198 GB (3,434,352,125 bytes)                 37
    Mailbox11 Syncing          146567                0 4.09 GB (4,391,621,840 bytes)                  50
    Mailbox12 Syncing          102574                0 6.194 GB (6,650,300,317 bytes)                 94
    Mailbox13 Syncing           25702                0 3.951 GB (4,242,569,651 bytes)                 59
    Mailbox14 Syncing           29478                0 3.448 GB (3,702,518,318 bytes)                 48
    Mailbox9  Synced             2874                0 4.444 GB (4,771,626,709 bytes)                 95


  13. The next morning, I was monitoring the status of the migration and saw this error:
    Get-MigrationUser -BatchId PublicFolderMigration | Get-MigrationUserStatistics | Select Identity,Status,SyncedItemCount,SkippedItemCount,BytesTransferred,PercentageComplete | Sort -prop Status,PercentageComplete | Ft -auto
    WARNING: The subscription for the migration user Mailbox2 couldn't be loaded. The following error was encountered: A subscription wasn't found for this user..
    Identity  Status  SyncedItemCount SkippedItemCount BytesTransferred                PercentageComplete
    --------  ------  --------------- ---------------- ----------------                ------------------
    Mailbox7  Syncing          128906                0 10.06 GB (10,804,408,110 bytes)                 20
    Mailbox10 Syncing          579557                1 7.109 GB (7,633,236,986 bytes)                  54
    Mailbox1  Failed          196706              176 9.482 GB (10,180,836,340 bytes)                 94
    Mailbox2  Failed            95156                0
    Mailbox11 Synced           488624                0 11.87 GB (12,750,050,794 bytes)                 95
    Mailbox9  Synced             2874                0 4.444 GB (4,771,626,709 bytes)                  95
    Mailbox12 Synced           102898                0 6.299 GB (6,763,569,925 bytes)                  95
    Mailbox14 Synced           104686                0 10.79 GB (11,583,208,485 bytes)                 95
    Mailbox13 Synced            65194                0 8.003 GB (8,592,870,685 bytes)                  95
    Mailbox4  Synced            23625                0 8.414 GB (9,034,298,415 bytes)                  95
    Mailbox3  Synced           219617                0 9.275 GB (9,958,462,334 bytes)                  95
    Mailbox5  Synced             1597                0 2.025 GB (2,173,804,244 bytes)                  95
    Mailbox8  Synced             6472                0 7.276 GB (7,812,204,276 bytes)                  95
    Mailbox6  Synced            57514                0 3.712 GB (3,985,266,258 bytes)                  95

    Mailbox2 had failed.  Ugh.  No one likes to see that, so you know what I did?  I checked to see details about the failure.

    Get-MigrationUser mailbox2 | Select -ExpandProperty ErrorSummary
    Couldn't find a request that matches the information provided. Reason: No such request exists.

    So, no move request exists? Awesome.  No idea how that happened; perhaps there was a database failover in the background and the underlying move request failed.  I decided to not give it too much thought and just restarted the migration batch.  This will not affect existing migration users (and their underlying move requests), but it will restart failed migrations.

    What about the other one?

    Get-MigrationUser mailbox1 | Select -ExpandProperty ErrorSummary
    Error: This mailbox exceeded the maximum number of corrupted items that were specified for this move request.

    176 items? Seems like we have a pretty low threshold for failure. The only way to resolve this error is to either A) remove the offending items from the source public folder or B) increase the error threshhold.  Not being in the mood to search for items to delete, I confirm the choice with my customer and run:

    Set-MigrationBatch -BadItemLimit 10000

    And then:

    Start-MigrationBatch PublicFolderMigration

    After a few minutes, I decided to check the status again:

    Get-MigrationUser -BatchId PublicFolderMigration | Get-MigrationUserStatistics | Select Identity,Status,SyncedItemCount,SkippedItemCount,BytesTransferred,PercentageComplete | Sort -prop Status,PercentageComplete | Ft -auto
    Identity  Status  SyncedItemCount SkippedItemCount BytesTransferred                PercentageComplete
    --------  ------  --------------- ---------------- ----------------                ------------------
    Mailbox2  Syncing               0                0 16.24 MB (17,031,786 bytes)                     10
    Mailbox7  Syncing          128906                0 10.06 GB (10,804,408,110 bytes)                 20
    Mailbox10 Syncing          609683                1 7.454 GB (8,003,508,841 bytes)                  55
    Mailbox1  Syncing          196706              176 9.482 GB (10,180,836,340 bytes)                 94
    Mailbox11 Synced           488624                0 11.87 GB (12,750,050,794 bytes)                 95
    Mailbox9  Synced             2874                0 4.444 GB (4,771,626,709 bytes)                  95
    Mailbox12 Synced           102898                0 6.299 GB (6,763,569,925 bytes)                  95
    Mailbox14 Synced           104686                0 10.79 GB (11,583,208,485 bytes)                 95
    Mailbox13 Synced            65194                0 8.003 GB (8,592,870,685 bytes)                  95
    Mailbox4  Synced            23625                0 8.414 GB (9,034,298,415 bytes)                  95
    Mailbox3  Synced           219617                0 9.275 GB (9,958,462,334 bytes)                  95
    Mailbox5  Synced             1597                0 2.025 GB (2,173,804,244 bytes)                  95
    Mailbox8  Synced             6472                0 7.276 GB (7,812,204,276 bytes)                  95
    Mailbox6  Synced            57514                0 3.712 GB (3,985,266,258 bytes)                  95

    Seems well enough.

  14. At one point, one of the migration requests failed.  I ran a Get-MigrationUser <Identity> | Select -ExpandProperty ErrorSummary, and was greeted with this very non-helpful one message:
    Error: There are 30 Public Folders that could not be mail-enabled. Please, check the migration report starting at 4/9/2017 8:13:15 PM for additional details. This may indicate that mail public folder objects in Exchange Online are out of sync with your Exchange deployment. You may need to rerun the script Sync-MailPublicFolders.ps1 on your source Exchange server to update mail-enabled public folder objects in Exchange Online Active Directory.

    No further details available (or so I thought).  I exported the report and looked in the Failures node.  After much poking around and asking my peers, one of them sent me a script to try and suss it out.

    $resultsarray = @()
    $mailpub = Get-MailPublicFolder -ResultSize unlimited
    foreach ($folder in $mailpub) {
      $email      = $folder.primarysmtpaddress.local + "@" + $folder.primarysmtpaddress.domain
      $pubfolder  = Get-PublicFolder -Identity $folder.identity
      $folderpath = $pubfolder.parentpath + "\" + $pubfolder.name
      # Create a new object for the purpose of exporting as a CSV
      $pubObject = new-object PSObject
      $pubObject | add-member -membertype NoteProperty -name "Email" -Value $email
      $pubObject | add-member -membertype NoteProperty -name "FolderPath" -Value $folderpath
      # Append this iteration of our for loop to our results array.
      $resultsarray += $pubObject
    # Finally we want export our results to a CSV:
    $resultsarray | export-csv -Path C:\Temp\mail-enabled-public-folders.csv -NoType

    I then took the output:

    $NoPublicFolderPath = Import-Csv C:\Temp\mail-enabled-public-folders.csv | ? { $_.Parentpath -eq "" } | Export-Csv C:\Temp\NoFolderPath.csv -NoType
  15. From here, I was able to see what Mail-enabled public folders had addresses but no path to send to.  Launch ADUC, view Advanced Features, and then browse to the Microsoft Exchange System Objects container.
  16. Right-Click and Add columns for E-mail Address and Exchange Alias--that way, if your organization is like my customer's and you have multiple objects with similar display names, you'll easily be able to figure out which ones you need to remove.
  17. Remove the objects from the Microsoft Exchange System Container.
  18. Re-run Sync-MailPublicFolders.ps1.
  19. Re-start the migration batch.
  20. At this point, there the error count went down, but there were still a number of problems (again, reported as super helpful "There are 10 Exchange Public Folders that could not be mail-enabled.").I'm sorry, but...Could you be any more vague?
  21. Another peer of mine pointed me towards this incredible script by Bill Long, and that was part of the secret sauce.  To use it, I had to get an export of the public folder tree from PFDAVAdmin and I included the following fields:PR_PF_PROXY
    PR_ENTRYIDBut in order to use PFDAVAdmin, I had to unlock the Public Folders.  So:

    Set-OrganizationConfig -PublicFoldersLockedForMigration $false

    And wait about 15 minutes.

    Then, I launched PFDAVAdmin, connected to the public folder tree, and exported the fields necessary.

  22. I ran Bill's script against that output file, re-ran Sync-MailPublicFolders.ps1 AGAIN, and now had synced set of public folders.  Thank you, tiny baby Jesus.  And then, just before cutting over, I decided to check the Get-MigrationUserStatistics.  Much to my dismay, I found the "There are 26 Exchange Public Folders that could not be mail enabled." error.  Again.  But I knew it was all good?
  23. Back to Sync-MailPublicFolders.  I noticed an error when running:
    4/12/2017 6:11:16 PM,1a3a8d9e-0eb6-4b8a-bf00-a305f3229b9e,Update,"Active Directory operation failed on CY1PR14A001DC06.NAMPR14A001.PROD.OUTLOOK.COM. The object 'CN=Staff Folder,OU=tenant.onmicrosoft.com,OU=Microsoft Exchange Hosted Organizations,DC=NAMPR14A001,DC=PROD,DC=OUTLOOK,DC=COM' already exists.","Set-EXOMailPublicFolder -OnPremisesObjectId:""ae9563f4-1056-44c1-846a-c948c771720b"" -HiddenFromAddressListsEnabled:""False"" -ExternalEmailAddress:""StaffFolder@domain.edu"" -Alias:""StaffFolder"" -EmailAddresses:@(""X400:C=US;A= ;P=ORG;O=Exchange;S=StaffFolder;"",""SMTP:StaffFolder@domain.edu"",""x500:/O=ORG/OU=EXCHANGE/CN=RECIPIENTS/CN=STAFFC89080BC4725C2AEEDFB74A5292C16AE6F7CEE"",""smtp:StaffFolder@tenant.onmicrosoft.com"") -Name:""Staff Folder"" -Identity:""CN=Staff Folder,OU=tenant.onmicrosoft.com,OU=Microsoft Exchange Hosted Organizations,DC=NAMPR14A001,DC=PROD,DC=OUTLOOK,DC=COM"" -ErrorAction:""Stop"" -WindowsEmailAddress:""StaffFolder@domain.edu"" -DisplayName:""Staff Folder"""

    What do you mean "The object already exists?"  I know it does--I'm trying to update it.

  24. The output file also contains the command it was trying to run.  I copy / paste it into a PowerShell session and after tinkering with the quotes  am able to replicate the error.  I start removing the parameters one at a time until I found the one that was causing it to fail--name.  Turns out, the customer had multiple mail-enabled objects with the same name field.  I confirmed this on-premises by running:
    Get-Recipient -anr "Staff Folder"

    And was presented with both a distribution group and a public folder bearing the same name attribute.

  25. I ran Set-MailPublicFolder -Identity <email> -Name "Old Name + PF" and re-ran Sync-MailPublicFolders.ps1.  Completed successfully.
  26. Restarted the migration batch.  Again.  And now, the world is at peace.

Completing the Migration

  1. At this point, it's time to finish off the beast.  From the On-Premises Exchange Management Shell:
    Set-OrganizationConfig -PublicFoldersLockedForMigration $True

    And wait.  Like about 10 minutes.  YMMV.  If you don't wait long enough, you'll receive this message when you go to run Complete-MigrationBatch in the cloud:

    Complete-MigrationBatch -Identity PublicFolderMigration
    Before finalizing the migration, it is necessary to lock down public folders on the legacy Exchange server (downtime required). Make sure public folder access is locked on the legacy Exchange server and then try to complete the batch again.
  2. I wait some more, try it again, and this time, get the prompt I'm looking for:complete-migrationbatch1Hells to the yeah.
  3. But alas, it was not to be.  Once more I received the "There are Exchange Public Folders that could not be mail enabled" error message, but to my utter dismay, the number had grown to 59.  It's like fixing one kind of problem revealed yet another.  This is literally the opposite of completing! The next step was digging yet further into the public folder migration logs, using a tip from another of my way-smarter-than-me coworkers.  You see, while the "Failures" log file entry showed me *that* there was a failure, the *Messages* entries were what I actually needed to find the problem folders.
    Get-MigrationUserStatistics mailbox1 -IncludeReport | Export-Clixml .\mailbox1.xml
    $Mailbox1Report = Import-Clixml mailbox1.xml

    Now, where I had looked at the Failures node before ($Report.Report.Failures), I looked in $Report.Entries.Messages this time:

    $Entries = $Mailbox1Report.Report.Entries | Select Message -Last 300 | ? { $_.Message -like "*could not be mail-enabled*" }

    The result? Lines like this:

    Public folder "/Schools/School1 HS/School1 Guidance" could not be mail-enabled.
    Public folder "/Schools/School2 ES/Staff News" could not be mail-enabled.
    Public folder "/Schools/School1 HS/Yearbook" could not be mail-enabled.
    Public folder "/Schools/School3 HS/Staff News" could not be mail-enabled.
    Public folder "/Schools/School4 MS/Media" could not be mail-enabled.

    Indeed, the error log did log the errors. I just didn't know where to look for it.

    These ended up being objects that had *some* attributes set for mail-enabled Public Folders on-premises, but not enough to actually be correct.  I quick run through the PF tree allowed me to "mail-disable" and "mail-re-enable" these public folders, which created the appropriate objects in the Microsoft Exchange System Objects container.

  4. And then again with the .\Sync-MailPublicFolders.ps1 from my customer's on-premises Exchange 2007 Public Folder server.
  5. And then again with the Start-MigrationBatch in Exchange Online PowerShell, which brought the mailbox objects to "Synced" state.
  6. And then again with the Set-OrganizationConfig -PublicFoldersLockedForMigration $true on the on-premises Exchange 2007 server.
  7. And again with the waiting.
  8. From Exchange Online PowerShell, I then ran:
    Complete-MigrationBatch -Identity PublicFolderMigration -SyncAndComplete

    And waited.

    For 7 hours.

    And then, I saw this:


  9. Now, just the tidying up:
    Office 365 / Exchange Online PowerShell

    Set-OrganizationConfig -PublicFoldersEnabled Local -RemotePublicFolderMailboxes $null
    Get-Mailbox -PublicFolder | Set-Mailbox -PublicFolder -IsExcludedFromServingHierarchy $false
    Set-Mailbox -Identity <test user> -DefaultPublicFolderMailbox mailbox1

    Adding SendAs:

    $SendAs = Import-Csv .\SendAs.csv
    foreach ($obj in $SendAs) 
        write-host "$($i)/$($SendAs.Count) adding $($obj.User) to $($obj.Identity)"
        Add-RecipientPermission -Identity $obj.Identity.Split("/")[2] -Trustee $obj.User.Split("\")[1] -AccessRights SendAs -confirm:$false; $i++

    Adding GrantSendOnBehalfTo:

    $GrantSendOnBehalfTo = Import-Csv .\GrantSendOnBehalfTo-Resolved.csv
    Foreach ($obj in $GrantSendOnBehalfTo)
        Write-host "$($i)/$($grantsendonbehalfto.count) Granting $($obj.GrantSendOnBehalfTo) Send-On-Behalf to folder $($obj.PrimarySmtpAddress)"
        Set-MailPublicFolder -Identity $obj.PrimarySmtpAddress -GrantSendOnBehalfTo $obj.GrantSendOnBehalfTo

    Exchange On-Premises Management Shell

    Set-OrganizationConfig -PublicFolderMigrationComplete $true
  10. Depending on how your Office 365 tenant is already configured, you may need to disable Directory-Based Edge Blocking (DBEB) in order to anonymous delivery to mail-enabled public folders to work.  DBEB doesn't see MEPF's as valid recipients, and will reject messages with a 550 5.4.1 "Recipient address rejected: Access denied." message.  You can disable DBEB by setting the Exchange Online domain to InternalRelay:
    Set-AcceptedDomain -Identity <domain> -DomainType InternalRelay

    I also ran into this issue: During the Batch Migration, you're instructed to create an internal relay domain for a well-known namespace:

    New-AcceptedDomain -Name "PublicFolderDestination_78c0b207_5ad2_4fee_8cb9_f373175b3f99" -DomainName tenant.onmicrosoft.com -DomainType InternalRelay

    However, you also need to add this to the Outbound to Office 365 Send Connector address space, or external mail destined for cloud MEPFs will go out the default send connector back to the internet and get rejected (if the customer has the MX pointed on-premises).  I'm putting a request in to have clarification made in our documentation to reflect that.


And the world rejoiced.

Comments (24)

  1. Turbomcp says:

    I’m glad I am yet to do this thing:)
    its not short:)
    as always , always interesting

  2. Joseph Moody says:

    Great write up and humorous to read! Thank you for putting this together!

    1. Thanks! Def Leppard and Chandler make everything more enjoyable.

  3. Peter says:

    Great write-up! Thanks for documenting this stuff, some of which I do myself in migrations, like the sendas and grantsendonbehalfto. Learned some things, and thought I knew everything when it comes to Public Folders and migrating them… Nice one!

    1. Thanks for the comments! I always learn something new every time I do this, too. I have put together a toolkit around planning and deployment for on-premises modern public folders as well as transition to Office 365, based on some of my larger projects. Most of my tools come out of particular tasks for customers that I adapt to be more general.

  4. This 52-step process is well documented, Aaron (and good humor throughout).
    Unfortunately, I think that most people will not survive it, nor have the time or patience to work through it.

    I’d love to get your take on our 4-step process. http://www.exchangesavvy.com/public-folder-migration-to-office-365/

    We have managed to make PF migration so simple that anybody can do it, as well as make it easy to cut over over and roll back the settings (especially around mail-enabled folders).

    1. I’m always interested to see what solutions may exist. In the last year, I’ve moved around 3 million public folders from Exchange 2007, 2010, and 2013 to other Exchange systems or Office 365 using the batch migration method described here, which is mostly online except for the actual cutover (just like hybrid mailbox moves). This method now supports incremental syncing to keep content virtually up-to-date for months or years, and uses the standard move request reporting to report underlying errors.

      The biggest challenge we face in consulting services is around multi-organization public folder trees for mergers and divestitures. As long as you understand the science around how many users will be accessing public folders and how you should be breaking apart the branches in your public folder tree to accommodate the necessary growth, I haven’t really run into too many problems. That being said, I’m always interested in learning about additional methods to solve complex tasks that we don’t necessarily have guidance around.

      1. Full Disclosure: I work with Geoff at ExchangeSavvy.

        Great blog post!

        The whole multi-org public folder problem is an interesting one, and one that PowerShell scripts are ill-suited to solve, IMO. Having an application drive the process allows a lot of logic to be easily introduced into the picture. The ExchangeSavvy solution is able to migrate and merge public folder hierarchies from multiple on-premise Exchange organizations into a single O365 public folder hierarchy. Part of this process is to allow the user to easily specify where in the O365 tree a given on-prem PF hierarchy should be rooted. This lets users organize the new O365 hierarchy however they see fit.

        A lot of the value of solutions like ExchangeSavvy is around handling so many of the complex details that you’ve documented. I think the average Exchange admin is really going to struggle to pull off a migration without either a tool like ExchangeSavvy or a seasoned pro such as yourself. Even for experienced guys like you, I think ExchangeSavvy offers a lot of value is certain situations where the MSFT scripts just don’t cut it, multi-org migrations being a perfect example. Other examples would be situations where only a subset of the on-premise folders are to be migrated (that’s super-easy to do with ExchangeSavvy) or you want to limit migration based on other criteria like item age (something else we can do).

        Aaron, if you ever wanted to take a look at ExchangeSavvy drop us a request at http://www.exchangesavvy.com. I think you’d find what we have interesting and I’m sure we’d learn a few things by getting to have you grill us on our solution.

        1. In the event that you want to migrate just a subset of folders, you can technically do that as well by editing the PF Mapping file output. 😉 No matter what solution you go through (native or on-premises), there are still a lot of steps to do prior to migration that need a human touch because they’re analytical (for example, if you have mismatched MEPFs, EntryIDs, missing MESOs or corrupt items in the PF Tree), as well as understanding how PF hierarchy works so you don’t end up with too few mailboxes to support your number of users (which can be disastrous).

          At some point, I’d be interested in looking at it, but since we have a working methodology that’s developed and supported by PG, it’s pretty hard to get a consulting services person to recommend a third-party product without voiding our own support. 🙂

  5. Contessa says:

    You really make it seem so easy with your presentation but I find this topic to be actually something that I think I would never understand. It seems too complicated and very broad for me. I’m looking forward for your next post, I’ll try to get the hang of it!

  6. Michael says:

    With Azure AD Connect 1.1.524.0 now supports synchronizing Mail-Enabled Public Folder objects , is it still required to use Sync-MailPublicFolder.ps1 in a hybrid environment (Ex2013)?

    1. There’s actually a different set of steps for doing this with Modern Public Folders, including the script “Sync-ModernMailPublicFolders.ps1.” I’m writing up a similar set of steps/procedures that you’ll be able to find in the new Office 365 Administration Inside Out book (http://aka.ms/o365adminio), but I’ll post a pretty comprehensive set here as well when I get it fully mapped out.

      1. Michael says:

        Thanks Aaron, looking forward to reading.

      2. The short answer is you have to disabling the MEPF sync in AAD Connect prior to completing these steps.

  7. Todd Huffman says:

    Thank you Aaron for posting with such great details, screen shots, logic for troubleshooting…Everything!
    Your Post helps understanding and walking thru the MS article like having a personal instructor!
    (Psst… Typo in Step 16 of the first section. “PFStatisticsNew.csv” doesn’t match Step 15’s output file “PFStatistics.csv”)

    1. Thanks! 🙂 Between steps 15 and 16, I had to re-run it, and I just forgot to note that the output for the second time running the command was “PFStatisticsNew.csv.” I’ll fix it. Thanks!

  8. Have you ever considered publishing an e-book or guest authoring on other websites? I have a blog based upon on the same topics you discuss and would love to have you share some stories/information. I know my visitors would enjoy your work. If you are even remotely interested, feel free to send me an e-mail.

    1. I’m always interested in guest opportunities.

      [ shameless plug ]
      I also have a book coming out that captures a lot of this and other things when moving to the cloud. You can pre-order it at http://aka.ms/o365adminio.
      [ /shameless plug ]

  9. Hey There. I found your blog using msn. This is an extremely well written article. I will make sure to bookmark it and come back to read more of your useful info. Thanks for the post. I will definitely return.

  10. Gareth says:

    Can you tell me what In need to replace in this part of the script to get it to run
    (Get-Mailbox ).legacyExchangeDN | Out-File .\MailboxLegacyExchangeDN.txt
    (Get-ExchangeServer ).ExchangeLegacyDN | Out-File .\ServerExchangeLegacyDN.txt
    $OAEndpoint = (Get-ExchangeServer).[0].ExternalHostNameHostnameString
    $OAEndpoint | Out-File .\OAEndpoint.txt

    1. Gareth says:

      Reason I ask I get a Missing closing ‘)’ in expression At line:2 char:3

      1. Looks like I had a typo. I’ve updated it. It should be:

        $OAEndpoint = (Get-ExchangeServer)[0] instead of (Get-ExchangeServer).[0]

        The end goal is you’re looking to save your external autodiscover hostname to a file, since you’re going to import it later.

  11. Matt Quinn says:

    Excellent guide, which came to me from a Exchange migration expert. I noticed you increased the Public Folders quota setting to 9.5 GB in the guide. I’d like to adjust settings to accommodate two large public folders – 14 GB and 7 GB. Should I change this quota setting to allow these PFs to migrate? Or should I break up the two PFs?


  12. Nathan says:

    Hi Aaron Guilmette,

    I was following your steps in migrating Public folder but I got this error code message in step 13.

    cd \PFScripts
    Mkdir C:\PFScripts\PFMigrate
    .\Export-PublicFolderStatistics.ps1 C:\PfScripts\PFMigrate\PFStatistics.csv publicfolderserver.domain.com

    Exisiting code:
    cd c: \pfmigration
    [PS] C: \pfmigration>.\Export-PublicfoldersStatistics.ps1 PFStatistics.csv FCIEXCH

    Error: A Parameter cannot be found that matches Parameter name server

Skip to main content