Migrate to SharePoint Online (O365–SPO) from On-Premise SharePoint or fileshare using new PowerShell migration APIs


     

    **Updated to include SharePoint On-Prem to Office 365 SharePoint Online Scenario

    Important

    • This documentation covers features that are currently in preview and not yet finalized for use in production code.
    • Replace <TENANT> with actual tenant prefix, <SPOSite> with SPO Site prefix and <On-PremSPSite> with SP On-Prem Site Prefix in all PowerShell cmdlets.
    • Triple-click in code box to copy Code / PowerShell cmdlets if they don’t fit on screen!

    Prerequisites

     To keep things simple, lets divide the entire process in few steps:  

    Step 1 - Create SPO Migration Package

    Step 1a – Create SPO Migration Package from SharePoint On-Prem Content

    • RDP to a WFE for On Premise SharePoint Server
    • Open SharePoint Management Shell as administrator
    • Export the Web / Site Collection using the below cmdlet
      Export-SPWeb -Identity "http://<On-PremSPSite>" -ItemUrl "/OnPremDocLib" -Path "C:\SPOnPremExport" -NoFileCompression -IncludeVersions 4

    • Note:
    • Copy Data: Copy the Contents of this folder to the location on machine which has SharePoint Online Management Shell installed. i.e. I copied contents from C:\SPOnPremExport from my SharePoint On-Prem Server to C:\MigrationPackages\SPOTemp on my laptop.

image

    OR

    Step 1b – Create SPO Migration Package from a file share

    Note:

    • OutputPackagePath folder hierarchy will be created if it doesn’t exist

    clip_image001

    clip_image002

     

    Step 2 - Prepare Final SPO Migration Package

    • Prepare data / files to be migrated using below cmdlet
    • For Step 1a
      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "C:\MigrationPackages\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOTEMP" -OutputPackagePath "C:\MigrationPackages\SPOMigrationPackage" -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -TargetDocumentLibraryPath "SPOMigration"

SNAGHTMLbecfed5

    • For Step1b
      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "\\nishant-lptp\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOTEMP" -OutputPackagePath "C:\MigrationPackages\SPOMigrationPackage" -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -TargetDocumentLibraryPath "SPOMigration"

    • There are six required parameters to enter (others are optional)
      1. TargetwebURL: points to the destination Web
      2. SourceFilesPath: points to the you want to migrate
      3. SourcePackagePath: points to your Temporary package folder
      4. OutputPackagePath: points to your final package folder
      5. TargetDocumentLibraryPath: the path to your destination library
      6. Credentials: SPO credential that has admin rights to the destination site
    • More Info: ConvertTo-SPOMigrationTargetedPackage - https://technet.microsoft.com/en-us/library/mt143609.aspx

    clip_image003

    clip_image004

     

    Step 3 - Upload SPO Migration Package to Azure Blob Containers

    • Follow the below steps to get Storage Account Name and two container names inside it. If you already have a storage account, you could use the same
      • Login to Azure Portal via https://manage.windowsazure.com
      • Browse to Storage > Create a new storage account e.g. nO365mig
      • In newly created storage account create two containers e.g. migration-files, migration-package
    • Prepare to upload the prepared data / files to Azure Blob Containers using below cmdlet
    • For Step 1a
      $azurelocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath "C:\MigrationPackages\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOMigrationPackage" -FileContainerName migration-files -PackageContainerName migration-package -AccountName nO365mig -AccountKey "REAPLCE_WITH_YOUR_AZURE_STORAGE_ACCOUNT_ACCESS_KEY"

    • For Step1b
      $azurelocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath "\\nishant-lptp\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOMigrationPackage" -FileContainerName migration-files -PackageContainerName migration-package -AccountName nO365mig -AccountKey "REAPLCE_WITH_YOUR_AZURE_STORAGE_ACCOUNT_ACCESS_KEY"

    • More Info: Set-SPOMigrationPackageAzureSource - https://technet.microsoft.com/en-us/library/mt143606.aspx

    clip_image005

     

    Step 4 - Submit Migration Job

    • Submit Migration Job to transfer data from Source <to> Azure Blob Containers <to> SPO Site Collection using below cmdlet
      Submit-SPOMigrationJob -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -MigrationPackageAzureLocations $azurelocations
    • Above cmdlet is generic and applies to both Step1a and Step1b
    • More Info: Submit-SPOMigrationJob - https://technet.microsoft.com/en-us/library/mt143605.aspx

    clip_image006

     

    Step 5 (Optional) - Check Migration Job Status

    • Check migration job (to transfer data from Azure Blob Containers to SPO Site Collection) status using below cmdlet
      Get-SPOMigrationJobStatus -TargetWebUrl https://<SPOSite>.sharepoint.com/

       

    clip_image007

    Step 1a

    image

    Step 1b

    clip_image008

     

    Step 6 (Optional) - Remove Migration Job

Comments (46)
  1. SharePoint Online Training says:

    Great article… Thanks for your great information, the contents are quiet interesting. I will be waiting for your next post.

  2. Charles turner says:

    if you have office 365 do you need to have a separate azure account? we have plenty of storage under available under office 365.

  3. Nilesh Talele says:

    Thanks a lot Nishant for the great article.

  4. @Charles: Yes, as I mentioned in requirements, valid Microsoft Azure subscription is required.

  5. Subhamoy Mukherjee says:

    Nice one Nishant

  6. Josh says:

    I can confirm that the method works beautifully. However, I seem to have a bit of trouble. I am trying to figure why it takes so long for the job to finish on azure before it shows up on the sharepoint site. Is this common? Should there be a significant
    delay? Maybe I'm missing something here?

  7. @Josh: Thanks, what is the size of Content being migrated and what is the approx. time it took? Is the status of job (Step 5) "Processing" when it takes time?

  8. Josh says:

    Since im just testing its about 8-12 mb. Processing takes about 10 seconds at most.

  9. ashish_1 says:

    @Josh, there could be a geo difference in your azure instance and O365 instance. For me, both in same geo location and it takes minutes to move MBs data to SPO lib.

  10. jeevan says:

    Thank you very much Nishant for this step by step explanation.
    Do you know how to migrate DocLib and each documents permissions?[CSV file logic]

  11. Senthil Kumar (Ex-MSFT) says:

    Wonderful article. Nishant I was working on this PowerShell and found document created and modified by are not getting carried over. Everything are under System Account. Do you aware of this issue and any workaround?

  12. @Jeevan @Senthi: Do you have User Mapping file in place in prescribed format as mentioned at
    https://technet.microsoft.com/en-us/library/mt143609.aspx?

  13. Ratnakar says:

    Hi,When i am migrating a site at ConvertTo-SPOMigrationTargetedPackage i got unrecognized GUID Format error.Can you help me in this case please

  14. @Ratnakar: Please explain how did you perform Step 1. Was it SharePoint On-Prem scenario? Did you specify -ItemUrl with Export-SPWeb?

  15. Ratnakar says:

    I tried Sharepoint OnPremise Scenario without itemurl parameter.Is it possible to migrate entire subsite directly into Sharepoint online ? I tried to take entire Subsite to Sharepoint Online.

  16. @Ratnakar: At this time, SPO migration pipe is oriented for list and document library content only. Hence, -ItemUrl is required.

  17. Binoj says:

    Hey When i am trying to migrate my sharepoint list to sharepoint online i am getting file not found error in step 2.What might be the reason do you have any idea ??

  18. jeevan says:

    @Nishant: Regarding properly resolving Author and Editor fields: I've proper linking to .csv file as mentioned in ConvertTo-SPOMigrationTargetedPackage. Do you have any sample of .csv file? What change should I expect in UserGroup.xml file after execution
    of this command?

  19. @Binoj: Please ensure you are doing step 1 correctly as mentioned in blog

  20. @Jeevan: I haven’t yet tested this approach, will test and update the blog when I get a chance.

  21. jeevan says:

    @Nishant: I am able to use CSV file user mapping approach. I was using wrong SSID values for AD users. With proper SSID values, I am able to migrate Author and Editor field values properly.
    Thanks.

    1. Ragnar says:

      Can anyone light some insight on the user mapping file internals. What is the correct SSID that I must use to put together the CSV and where do I get it?

  22. @Jeevan: Thank you for the update.

  23. Fresher says:

    Hey, 'Parameters specify a Document Library but manifest does not contain a matching Document Library node' during step 2.. what error is this about. Do you have any idea.. Please help.

  24. @Fresher: Please detail out your step 1.

  25. Fresher says:

    Hey.. Sorry.. I gave a List in '-ItemUrl' in step-1 and was trying with '-TargetDocumentLibraryPath' in step-2. Just saw the '-TargetListPath' parameter and tried with that and it worked.

  26. Gopi Krishna says:

    Hi Nishant,

    To getting packages (xml files)
    is there any other alternative for "Export-SPWeb" command or can we generate these (8 xml packaged files) from the content database. because we don't have a permission on SP server 2010.
    By Server object model (using Microsoft.sharepoint.deployment.dll) i am able to generate these packages.

    my requirement is we need to migrate data from on-prem to O365 site using your concept (Azure pipeline).

    Thanks in advance.

  27. Fresher says:

    Hey.. I used my office 365 import storage account for both -FileContainerName and -PackageContainerName and also as -AccountName in step-3. But while submitting spo migration job in step-4, i received an error as ' The specified Azure source file and manifest
    containers cannot be identical'.
    Can I use my office 365 import storage account instead of azure storage account..? If so what error is this about.. Please help.

  28. @Gopi: You will need to use Export-SPWeb. There is no other alternative that I know of…

  29. @Fresher: 1) The error is self-explanatory; you need to use two separate containers. 2) As I mentioned under prerequisites section of blog, you will need Azure Account.

  30. Fresher says:

    Ok dude, I have an azure account and created a storage with two containers. I mentioned all of them in the $azurelocations. But I was wondering how will I provide authentication for the account. Coz, when i submitted the migration job I got error as 'Access
    denied. You do not have permission to perform this action or access this resource'.
    There was one parameter -Credential in Submit-SPOMigrationJob. So I thought I could provide authentication for azure account through that. But that gave me an error ' The Login server cannot issue the requested compact encrypted ticket because a Data Encryption
    Key (DEK) has not been uploaded to the site'. I could not recognize this error.. Please help.

  31. Ana says:

    Hi Nishant,
    Great post! but if I understand correctly this method is only available for SharePoint 2010 or SharePoint 2013 On Premises, what about SharePoint 2007?, the stsadm operation Export does not have an option for library or list.

    Thanks in advance.

  32. @Fresher: Please open a support case.

    1. Fresher says:

      Hi Nishant,

      Now I understood the commands and working fine.. Thanks..
      I’m eagerly waiting to know if you guys are working on the api that helps to migrate the whole site and when it would be out for usage..

  33. @Ana: Correct, this method is available only for SharePoint 2010 or SharePoint 2013 On Premises like you mentioned. SharePoint 2007 will need an upgrade to SharePoint 2010 / SharePoint 2013 prior to SPO migration.

  34. Hi Nishant,

    Can this migration approach be used for migrating content from one SPO site collection in one tenant to another SPO site collection on another tenant?

  35. I have tried migrating from SharePoint On-Premise 2010 and 2013 and also from File Folder data to SharePoint Online, all steps gets completed without any error, however we can't see any migrated data and migration job gets completed in few seconds. something
    is going but not sure what.
    Is it because I am using trial version of both Office 365 and Microsoft Azure?
    It would be great if you help me fix the issue.

  36. @Pankaj: Do you see the data copied to Azure containers after completing the steps?

  37. @EValliarampath: I don’t think so. Once you have SPO Migration Package created, you can proceed further however I dont see a way to create SPO Migration Package for content which resides on SPO.

  38. MSFT_SE says:

    Hi Nishanth, trying to migrate about 30-40k items (approx 15 GB). Not all files have moved to SPO (though all moved to Azure storage successfully).
    What could we check in this scenario?

    1. Check the Migration Job status…

  39. Jimmy Brothers says:

    Hi Nishant. Thanks for the article. I am trying to use this method to move some files into OneDrive for Business. When doing the last step to submit the job it is throwing this error:
    [12/23/2015 12:53:24 PM] [Folder] [Documents] [Documents] [a034d318-3d59-418b-a3b4-72c5a56fbcf5] [Error] [-2130247142] [Microsoft.SharePoint.SPException] [Folder “Documents” does not exist.]
    [12/23/2015 12:53:24 PM] [FatalError] [-2130247142] [Microsoft.SharePoint.SPException] [Folder “Documents” does not exist.]

    I had set the targetDocLib as: $targetDocLib = ‘Documents/st0106’

    Thanks for any insight you may offer,
    Jimmy

  40. Hiran says:

    Very nice article. This really helped me to do SharePoint migration. Thanks alot

  41. Neil Phelan says:

    Nishant this is a great article and I have created a script file and it just works every time fro small data migrations. However more recently I am coming across larger data migrations resulting in not all data been transferred.
    I noticed errors in the copymigrationpackage and createmigrationpackage texts files. Can you advise the importance of checking these and can I assume that if there is any errors I will get data transfer failures. Or can you reference me to any documentation regarding these files. Thanks Neil.

  42. Laurence73 says:

    Nice work just please edit Step 2. -SourceFilesPath should be same as in Step1 -“\\nishant-lptp\SPOTEMP” and not

    “C:\MigrationPackages\SPOTEMP”

Comments are closed.

Skip to main content