Migrating offline files (CSC) using USMT 4.0

One of the questions that came up after my TechEd New Zealand session on USMT 4.0 was whether USMT migrated the contents of the client-side cache (CSC) used for offline files.  Well, it sounds like it “sort of” does – but by default, it only moves the “dirty” files (those not yet sync’d to the network location).  That’s a decent default I suppose, as the remaining files can be pulled back from the network after the state is restored, and the modified files won’t be overwritten.  So there’s no data loss (always a good thing), but there will be extra network traffic to pull the content down to the cache again.

The actual cache migration is performed by a plug-in to USMT, so the question is whether that plug-in can be influenced to capture everything, instead of just the “dirty” files.  From http://support.microsoft.com/kb/942960, you can adjust the behavior by telling CSC you want to migrate everything.  (While this article talks about MigWiz.exe, the Windows Easy Transfer Wizard, the underlying engine being used is basically the same as that used by USMT.  So the end result of setting “MigrationParameters” should be the same.)

But before you say “great, let’s do it” you need to understand what’s going on behind the scenes.  First, this CSC migration plug-in is called automatically by USMT as part of the Windows manifest processing.  If you search through the Scanstate log you’ll see lots of references to it, with “CscMig” in the log entries.  For example, here is an entry I saw on my computer for a “dirty” file (one that was created while the folder was offline, so USMT needs to capture it):

2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item Created while offline (dirty).txt continueCtx = 00000000003DC480

But the rest of the files in that folder were skipped, as I didn’t have the MigrationParameters registry key set:

2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractFile(653):Skipping item File1.txt because it is in sync with remote location. ItemStatus = 00050020

OK, great, we see what’s happening.  But it’s also worth digging a little deeper and seeing what it did with that original file (something you can see from specify verbose logging, /v:5):

2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigGetWorkingDirectory(837):exit: workingDir = <\??\C:\Users\mniehaus\AppData\Local\Temp\tmp6865.tmp\Working\agentmgr\CCSIAgent\CSC>, status = 0x00000000 ( EE = 0 )
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item \\ continueCtx = 00000000003DAF00
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item bdddev continueCtx = 00000000003DB9A0
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item data$ continueCtx = 00000000003DC480
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item Created while offline (dirty).txt continueCtx = 00000000003DC480
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigWrite(446):exit: bytesWritten (743) at offset (0)
2011-08-24 22:53:43, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractFile(974):Backup API file content till offset 743

So it used the Windows Backup API to make a backup of the “dirty” file in the CSC cache and placed that backup into a folder in my %TEMP% directory named “tmp6865.tmp\Working\agentmgr\CCSIAgent\CSC”.  So even though I’ve specified to use hardlinks, there is data copying going on.  In the end, this temporary folder created by CscMig is included in the hardlinked state store, but because these are different files (the backups) you will see twice the disk space consumed, so if you have lots of cached data you better have lots of free disk space to store the data.  This isn’t so bad in the default configuration where it is only grabbing “dirty” files, but if you tell it to backup all files, then what happens?

No better way to find out what will happen than to try it.  I made the MigrationParameters registry entry as described in the KB article mentioned above and repeated the Scanstate execution.  (No service restart was required.)  Upon checking the log, I can see now that each file in the CSC was backed up into the temporary folder.  Where before it said “skipping item File1.txt”, now it says it’s backing it up:

2011-08-24 23:07:54, Info                  [0x0808fe] Plugin {0db12ccb-7cfd-46b6-b4d1-daa6ff0fbcf7}: CscMig: CscMigpExtractItem(1124):enter: Processing item File1.txt continueCtx = 000000000045C480

And like before all of these items get backed up into a temporary folder, then that temporary folder is hardlinked into the state store folder.  So hardlinks or not, if you had 2GB worth of cached files, you’ll end up with those being doubled (temporarily, until the process is complete and the temporary folder and state store are cleaned up).  It’s actually going to be tripled if you aren’t using hardlinks:  first copy is the original file, second copy is the backup, third copy is contained in the compressed state store.

So that definitively answers the question of whether you can get USMT to migrate the complete contents of the client-side cache.  It may not answer the question of whether you should do that, but hopefully the information is useful to help you make that determination yourself.

Comments (3)

  1. Kevin says:

    Thanks for this. I will ask our engineers to try but in the meantime… is changing the location of %TEMP% an option to save on C: drive space? e.g. to a temporary USB drive? It may be slower but may allow migration to proceed where users don't have sufficient capacity.

  2. showbox says:

    Thanks for the great info. I really loved this. I would like to apprentice at the same time as you amend your web site, how could i subscribe for a blog site?
    For more info on showbox please refer below sites:
    Latest version of Showbox App download for all android smart phones and tablets.
    http://movieboxappdownloads.com/ – It’s just 2 MB file you can easily get it on your android device without much trouble. Showbox app was well designed application for android to watch movies and TV shows, Cartoons and many more such things on your smartphone.
    For showbox on iOS (iPhone/iPad), please read below articles:
    Showbox for PC articles:
    There are countless for PC clients as it is essentially easy to understand, simple to introduce, gives continuous administration, effectively reasonable. it is accessible at completely free of expense i.e., there will be no establishment charges and after establishment
    it doesn’t charge cash for watching films and recordings. Not simply watching, it likewise offers alternative to download recordings and motion pictures. The accompanying are the strides that are to be taken after to introduce Showbox application on Android.
    The above all else thing to be done is, go to the Security Settings on your Android telephone, Scroll down and tap on ‘Obscure sources’.

  3. aw says:

    hai, I just want to tell you that I am just very new to blogs and seriously loved this website. More than likely I’m planning to bookmark your blog post .
    You amazingly come with really good posts. Thanks a lot for sharing your blog Microsoft.