MAT (powered by Project Shift)

Can’t we convert faster?

The most common complaint I hear when working with customers who are converting VMs from a VMware platform is that it’s slow. Now for many customers technologies like MVMC or SCV2V are great. They are easy to use and provide relatively quick results. These customers have some flexibility in their migration windows and they can tolerate a bit of downtime.

Of course, not everyone has that much flexibility. Large enterprises and hosters often have smaller service windows to work within. For many of them just the time it takes to copy the VM from the VMware datastore over to the target is too long and risks their SLAs on downtime. In fact I’ve worked with a few who have had maintenance windows as small 15 minutes. That’s crazy small and the fact is, tools like MVMC just can’t convert most VMs within a timeframe that small. The original MAT allows us to convert multiple machines at the same time but it doesn’t alter the speed of the underlying conversion engine.

The simple fact is that most conversion technologies are fighting physics. The VM needs to move from VMware’s storage onto a platform that can convert it, then it needs to get to the Hyper-V destination – so basically it has to be copied twice. Of course, there are limits to how fast a customer can move bits across their network. That speed limit multiplied by the number of bytes is the best conversion time they can manage. Even warm conversions like Double-Take Move have this liability; the difference is that with a warm conversion tool the copy happens while the VM is running and without downtime. Cold migrations perform their copies when the VM is off.


The opportunity

Not long ago I was watching NetApp show off their conversion technology, dubbed internally at NetApp Project Shift. I even mentioned it in my previous blogs.

What Project Shift does is convert VMs located on a NetApp controller between formats at amazing speeds. It can convert between several formats but I was obviously most interested in the VMDK to VHD. For example, this Project Shift can take a 40GB VMDK file and convert it to a VHDx using their cmdlet “ConvertTo-NaVhd” in about 5 seconds. Crazy right?

The trick here is that they don’t have to move the data, it stays on the controller. More than that, they don’t even need to copy the data on the controller. Their cmdlets use NetApp FlexClone technology to create a virtual copy of the VMDK that consists of a minimal amount of metadata (pointers to existing data blocks). The NetApp cmdlet simply Clone the Data from the VMDK into a VHD/VHDX writing the appropriate metadata as it goes. The resulting file is a VHD or VHDx file that takes up practically no extra space on disk. It’s like magic made out of chocolate.

In terms of a full conversion, the NetApp cmdlet just converts the disk format from VMware to Hyper-V. It doesn’t actually create a new VM. For a complete conversion, you need to collect details about the VM, remove VMware Tools and of course create a Hyper-V VM to which uses that new VHDx file. That sounds like a lot of work for an average customer to go off and script. Sigh.

So I went back to blogging about the MAT (Migration Automation Toolkit) with management and workflow features, each as rich and fantastic as peanut butter and I wondered wistfully about Project Shift and the chocolaty magic it performs.

And then it happened, we had our peanut butter cup moment. What if we took specific portions of the conversion capabilities from NetApp’s Project Shift technology and mixed it with MAT? Two great technologies that work better together.

Much goodness followed.


What we ended up with, we’re calling MAT (powered by Project Shift), a cold conversion technology that operates at warm conversion speed. We can convert most VMs with less than 5 minutes of downtime (and most of that it waiting for VMware tools removal).

The technology preview of MAT (powered by Project Shift) is available today!

Now this only works on NetApp FAS controllers (running Data ONTAP 8.2 or later) so you can’t just run this in any environment.

How it works:

I’ll post more in the coming weeks but we certainly need to start out with some basics around how all this works.

What you need:

As you can see from the architecture diagram below this overall design is very similar to MAT. Much like with MAT you simply need to configure the Variables.xml file with your details and you can run the script (fair warning there are quite a few more variables now).

The technology preview also assumes that your VMs are already stored on the NetApp FAS controller (if not, a simple command like “Get-VM $VMName | Move-VM –Datastore $NetAppController” can perform a Storage vMotion and get the VMs where they need to be).


To get started, after you have your variables set, you go through a few steps:

  • ./Convert-VM.ps1 Configure-NetApp
    • This will configure your NetApp controller so it is talking to both Hyper-V and VMware and the configuration is correct for the conversions
  • Move your VMs to the new container you just created
  • ./Convert-VM.ps1 Collect
    • This will collect your VMware information (like in the MAT)
  • Update the database with the VMs using the VMList.txt (like in MAT)
  • ./Convert-VM.ps1 Convert
    • Starts the actual conversion (like in MAT)

The version we have available today is a technology preview and any NetApp customer is free to test it.

The Dream Team

The MAT (powered by Project Shift) is the result of my collaboration with two guys with a can-do attitude and I want to thank them here:

Glenn Sizemore (NetApp). Glenn has been preaching the awesomeness of NetApp’s Project Shift and was critical in helping get the disk conversion logic into the toolkit. He’s also more than a little handy with PowerShell, I think he may be gunning for Jeffrey Snover’s job. A number of general improvements to the layout and design of the MAT as a whole are all thanks to his suggestions.

I also want to especially thank NetApp for being as generous with Glenn’s time as they have been. Sharing a resource as important as Glenn is a rare thing.

Michael Greene (Microsoft). Michael Greene has forgotten more about networking than I know. His primary focus was enabling the toolkit to collect information about the networking details for each VM and making sure they were recreated on the other end. The dude is smart. He would suggest up solutions quicker than we could test them. When he hands you a chunk of encoded PowerShell (which looks like gobbledy goop), just run it. Watch what happens – you will be amazed.