I was chatting with Brian Bourne from CMS Consulting in Toronto about some Hyper-V R2 work that they have done recently. In the conversation Brian shared a ton of learning's from the field on some of the Hyper-V R2 upgrades, deployments and V2V/P2V migrations. I asked him to write up a post and he did, so much so that I had to split it into two posts. You can find Part 1 here!
The release of R2 has been a major step forward for Microsoft’s virtualization strategy. It has also meant a rise in customer interest and willingness to move their data center and production servers on to the platform. Here at CMS Consulting, we’ve been offering both training and consulting services to help customers through the process. Here are some of the things we’ve learned along the way.
Virtual to Virtual (V2V) Migrations
In theory, this should be easy as pie right? The machine is already virtualized. How hard can a conversion be? Once again think about cloning to new hardware. For Virtual Server or VMWare to Hyper-V migrations, it truly appears as a completely new hardware platform to the operating system. This leads us to a few tips I’d like to add to the standard documentation.
- Cleanly shutdown the virtual machine, merge snapshots if they exist.
- After migration, logon to ensure the Hyper-V integration components are installed and working. VMM may think its ok, but it may not be. More tips on this below.
- If you are migrating offline or inactive virtual machines, remember they do get started as part of the migration process. This means you better think about the consequences of having that machine come online and what network its virtually “plugged in to” when it does.
Overall V2V migrations from Virtual Server and Hyper-V do go smoothly, but realize that the machine will be stopped and offline for the duration of the migration. Online migration is not a listed option. Data copy and fix-up time will determine the outage duration. Also, plan additional outage time so you can manually clean up and test the server. You may wish to have a strategy to block user access to services until you’re confident you want to introduce the server back into the environment.
If you want to do a migration “online” you will need to treat the VM as if it was a physical machine and follow the steps for P2V online migration. In fact, my team has used this strategy a number of times for migrating difficult VMWare virtual machines. Success when migrating a VMWare source varies widely based on ESX version, complexity of “hardware” configuration under ESX and other environmental factors. Sometimes treating the source as a P2V really is the best option.
Physical to Virtual
What I find odd about the VMM wizard for physical machine migrations is that it assumes you want to do an online migration and actually hides the tab for offline options from plain sight. Sure, I trust VSS enough for backups – but if I’m going to take a production server and virtualize it, I’d like the data to be as stable and consistent as possible. So for my money, I do an offline P2V if at all feasible. There are a few scenarios where you might forego an offline migration. Here are some scenarios where I would do an online P2V:
- You just want a copy of the server for testing or giggles, so data consistency doesn’t matter
- You truly can’t afford the server outage while data copies
- You have a source server with less than 512MB. (I recently had to migrate an old server with 448MB - online worked but offline won’t even start.)
- Finding drivers for WinPE to boot on some old hardware appears to be more effort than it’s worth.
I’d like to also suggest than when you’re looking at doing a P2V you also consider the “build fresh” strategy. This could be your opportunity to upgrade to Server 2008 R2 as the operating system and consolidated roles and services. There are two reasons to do this. First is to take advantage of the new features and increased performance of R2. Perhaps more importantly is to reduce the number of overall machines in your environment. I don’t mean physical machines here. I mean machines you have to license, patch, monitor and otherwise manage. If you can consolidate roles and services to a single VM then now is a good time to do it. Let’s not forget that fewer VM’s also means a reduction in hardware requirements and those ever-important spindle counts.
I also want to talk about P2V of non-domain machines. If you read the documentation on Technet, it very clearly states that the source machine needs to be either a member of the domain or there must be a domain trust. I have found this to be categorically untrue. I’ve had no issue doing P2V with both workgroup and isolated domain machines (including the DC of an “isolated” domain). The trick was to make sure the source machine had connectivity with both the VMM server and the target Hyper-V host. Once that was sorted out, I simply entered the appropriate machine credentials in VMM and everything worked fine. In theory you could also use the SysInternals Disk2VHD utility. This tool is designed for online use only. Although I previously recommended against online migrations, I started thinking about using Disk2VHD while booted to an alternate OS. We did some basic testing and found it won’t run under WinPE. In theory you could cobble together a full Win7 boot from USB and make it work. If we ever get around to that, we’ll post the results. In all cases so far, it’s easier to move everything on to a temporary network switch, P2V and then move back to the appropriate networks.
Before attempting a P2V, try to get the source machine “as clean as possible”. Ensure the source meets the requirements for free space and service pack levels, run a checkdisk and defrag on all partitions and remove unused programs and drivers. You will also want to remove hardware-vendor specific management tools. If you are doing a P2V migration against a virtual machine, be sure to remove VMWare tools, or the Microsoft Integration components. Stop all non essential services and applications. If you find that the P2V task is failing in the SCVMM scan, then start by looking at what security products you have installed. The P2V agent is installed at the time you click “scan system”. I have had it hang with no errors on either the SCVMM server or the source physical machine. In my case, the VMMInstallDetector service was hanging, and the culprit turned out to be the anti-virus product on the server. You can try to manually run the agent installation from “\\vmmservername\c$\Program Files\Microsoft System Center Virtual Machine Manager 2008 R2\agents\p2v\” if you’d like a closer look at what’s happening.
Here’s one last thought on P2V. I always choose the option to shutdown the physical machine after conversion. I don’t want the machine restarted. As soon as it restarts, I know my virtual machine and physical machine are no longer the same and who knows what might change, update or replicate when the physical machine starts. So I want to make sure it stays offline and the machine continues its new life as a virtual machine. There’s one catch. This strategy will result in the overall conversion job ending in a warning state with error 458. The warning essentially tells you that because the machine shutdown, the VMM agent didn’t get removed. If for some reason you decide to bring the machine back online – remember to manually remove it.
Some Hints for All Migration Scenarios.
There are a few things that SCVMM “fix up” doesn’t consider that you will need to.
- Windows will need activation again. Just like when you clone a machine and there’s a substantial hardware change – a VM migrate will trigger re-activation.
- Personally, I’m fussy about my machines looking for hardware that doesn’t exist. It just slows down boot time and I believe, overall machine stability. If you follow the instructions in KB241257 you’ll be able to see all the phantom devices in device manager on all versions of Windows (not just 2000 as the KB suggests). Delete these non-present devices.
- Near every physical machine I’ve ever converted starts with an ominous “Service Failed to Start” message. A quick look at the event viewer shows EventID 700 –The Parallel port driver service failed to start due to the following error”… the quickest way to make this problem go away is to change HKLM\SYSTEM\CurrentControlSet\Services\Parport\Start from a value of 3 to a value of 4.
- Various other products might complain, thinking they’ve been moved, so you should test everything. A perfect example is a terminal services licensing server will need activation again. If you don’t think to look at this, then 90 days from now you can expect user calls.
- On the odd occasion, when migrating 2008 servers the HAL may not get switched when it is supposed to be. The symptom isn’t obvious. Integration components will appear to be fine. You’ll run and re-run integration services setup and it will install successful but simply not work. This means no mouse control in the virtual machine which will add to your frustration if you are using remote desktop to connect to the VMM server or Hyper-V Manager. Here’s the trick. On the VM open the system configuration utility (MSConfig.exe). Click the Boot tab, and then click Advanced options. Select the Detect HAL check box, click OK, and then restart the virtual machine. You’ll find that your integration services will now magically start working.
- Time synchronization is a funny thing. It’s ok to have VM’s sync time with the host, if the host syncs with a domain controller (PDC emulator) but if you’ve gone and virtualized your PDC emulator you’ll be in for an entertaining circle of lost time. Don’t have your PDC emulator syncing time with the host. (See reference links below).
I’d suggest that if you were hoping to point a wizard at your server VLAN, cross your fingers and click “Next, Next, Finish” – then you’re probably going to be disappointed. The migration to Hyper-V needs to be approached with the same planning and consideration you would give to any server refresh or move. With a cautious approach and a maintenance window that will allow for testing and troubleshooting time, you will be ensured success.