Murphy Loves Migrations: USMT and the Case of the Missing Folders

Article
07/05/2011

Hi folks, Ned here again. You may have heard of a corollary to Murphy's Law: "Anything that can go wrong, will - and at the worst possible moment " Whether it comes to carrying the on-call phone, installing a service pack, or running a huge client migration, Murphy is there to kick you right in the neck.

Today I talk about a scenario where USMT fails to migrate certain folders on the root of the drive. It demonstrates how to detect and fix this specific issue. Moreover, it again shows how diverting from default to be pointlessly unique can have serious unintended consequences.

The scenario

I'm migrating from XP to Windows 7 and everything is going well. Too well… One of the business units testing their migration reports that certain folders are not migrating. Other test groups aren’t seeing this issue, and even the group with problems is not seeing all of their user data folders fail to migrate, only one or two. It turns out it's always the same couple of folders - one is called c:\docs and another is c:\data.

A little more digging shows that in my test lab, these folders and their contents migrate without issues. In addition, it turns out that some other business units have these folders and they migrate fine too. So what is it about these particular computers?

The investigation

I start by examining the scanstate.log (making sure that it was created with option /v:5) - it will tell me two things:

1. What XML was provided on the command-line to scanstate.exe?

2. Were they any errors or indications as to why these folders were skipped?

We need the XML provided to scanstate as it's the most likely cause of the problem. After all, it's designed to include and exclude data, and it's likely that someone has specified that these folders should not migrate. That would also explain why only one group is seeing the issue, and only for certain folders. For example, if I saw this log, I'd want to see engdrafts.xml and understand what it's doing.

If I were the extremely suspicious type, I’d also want to see the migdocs.xml and migapp.xml to make sure they are factory fresh.

It turns out though that the XML above is blameless. So I examine the log, looking for error and warning. And I find some! Yay! But they are expected and never mention my folders! Boo!

So that leaves me with looking for mention of the folders themselves, irrespective of problems. And interestingly, I only find these innocuous references:

What the heck does found application root mean? I don't see them in my own repro and not in the working business unit's either.

The reason

The underlying migration engine that USMT calls into intentionally generates exclusions for application folders. Typically, these are under Program Files so no one cares - after all, when was the last time you saved a document within the MS Office folders?

Definitely not here. Sheesh…

However, certain apps may be custom installed on the root of a drive (for instance, Oracle and Lotus Notes). When gathering the list of data to migrate, USMT calls a series of "docfinder" functions that examine registry keys for folders and binary file references. In order to prevent partial migration of applications to the destination folder, any applications that have registered their installation folders in the following locations are automatically excluded by scanstate:

HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\App Paths\*\ [Path]
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\*\ [Path]
HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*\ [InstallLocation]
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*\ [InstallLocation]
HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Installer\Folders\*
HKLM\Software\Microsoft\Windows\CurrentVersion\Installer\Folders\*
HKCU\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*\ [InstallLocation]
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*\ [InstallLocation]
HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\SharedDlls\*
HKLM\Software\Microsoft\Windows\CurrentVersion\SharedDlls\*
HKLM\SOFTWARE\Wow6432Node\Classes\CLSID\*\InProcServer32\ [Default]
HKLM\SOFTWARE\Classes\CLSID\*\InProcServer32 [Default]
HKCU\Software\Microsoft\Windows\ShellNoRoam\MUICache\*

No modern Microsoft application should register itself in these folder root locations by default, as that is not a best practice; they will all have their core folders in c:\program files or c:\program files (x86). They could have been custom-installed by an administrator on the root though.

So I examine the registry on one of these computers that refuses to migrate those folders and lo:

Sonofa! Someone installed two applications on the root of the drive, but used directory names that encouraged users to come fill the folders with personal files (in one real world case it was c:\documents and c:\downloads). Therefore, that simple message now makes sense:

Info [0x000000] Found application root: c:\docs

All by design, naturally: the design of Murphy.

The solution

Fortunately, there are a couple of easy fixes for this issue.

Override the behavior with an <include> for the folders that are being skipped.

Use a simple <include> rule in your XML that covers these folders and migrates them regardless of the migration engine's built-in rules. Many samples of this in article Include Files and Settings.
The downside to this solution is that application files will migrate over in addition to user data. You may also want to create some <exclude> XML for known "bad" file types like EXE, DLL, OCX, etc. using article Exclude Files and Settings.

Locate the registry entries for the application that is using a root folder and remove them prior to migration

If the computer is being wiped, the application registry entries are no longer important on the machine running scanstate so they can be deleted.
If the values are orphans of an old application that no longer exists, they can be deleted.
If the computer is going to be re-used without reloading, alter the registry values to no longer point to the root folder that contains user data. Then put them back when done. Use REG.EXE and a batch file that wraps the scanstate.exe or investigate more sophisticated scripting.

What if this wasn't the problem?

If you're not seeing the "Found application root" log entry, but are still missing folders, this leaves two likely causes:

1. Empty Folders - the migdocs.xml does not migrate empty folders on the root. If you are just comparing before and after, make sure the folder has some actual contents. Don't use an empty folder as your test or you will run in circles. :) I usually just throw a TXT in there, although sometimes I lovingly craft a canary file.

Now that is worth migrating

2. Those custom XML excludes - I mentioned them above and that I didn’t find anything that looked like an exclude in this scenario. Go back and look harder! Remember that it may fall under a wildcard like c:\* or c:\somefolder\someotherfolder\*. You can use MIG_ENABLE_DIAG to help when all the code starts to blur together…

Finally

I have another Murphy's Law variant for you and I'm sure the experienced readers know this one. I call it “Ned’s First Law of USMT”:

The chance of a computer having migration issues is directly proportional to the pay grade of the end user.

If you don't plan to get backups for every computer you migrate - and you really should - at least get them for the Executive laptops. Their data isn't more important, but their opinion on the migration probably is. You would be staggered at the number of cases opened here for data recovery on the CEO's laptop, where no one got a backup before they scanned the data and wiped the computer, only to find that the store file was corrupted by some piece of $#%^& USB thumb drive. Don't join the ranks of the unemployed: get a backup.

Until next time.

Ned "the perversity of the Universe tends towards a maximum" Pyle