Step-By-Step: Using Custom VM Images and Resource Groups in Microsoft Azure – Blain Barton

Article
09/12/2014

Part 21 - Modernizing Your Infrastructure with Hybrid Cloud – Using Custom VM Images and Resource Groups in Microsoft Azure

In this article, I’ll start by exploring Custom Virtual Machine (VM) images and conclude my write up with Resource Groups within the Microsoft Azure Preview Portal.

I hope you’ve had a chance to read Part 18 - “Migrating Physical Servers to Virtual Machines” in the Modernizing Your Infrastructure with Hybrid Cloud Series.

Part 18 explains how to convert or migrate a physical server from your private cloud on-premises infrastructure and bring it into Hyper-V Manager. The Microsoft Azure Pack along with System Center Virtual Machine Manager (Private and Public Cloud integration with AppController) are great hybrid tools as well as pushing your workloads to Microsoft Azure.

When you’re considering workloads in the public cloud, Microsoft Azure offers a couple of ways to get your images created and functional on Microsoft’s hardware. Why bother with managing the hardware and instead just pay for the services and manage your workloads from an RDP session. The cloud is only as good as your internet connection. Keep in mind, it’s not the connection that Microsoft provides you to connect to your server workloads, it’s your ISP that delivers that service level agreement. I should know, I used to live on a 1200-baud modem connected with Spry’s Internet-In-A-Box which was one of the first commercially available Internet connection software packages available for sale to the public. The good thing is I started with computers after punch cards! And with the lightning speed of technology we now have things like a Terabyte of storage inside a Swiss Army Knife produce by Victorinox.

System Center Virtual Machine Manager and AppController are great Hybrid Cloud tools that help Administrators manage the way you might integrate your traditional private cloud workloads to integrate with your public cloud services.

Using Custom VM Images in Microsoft Azure

Once a custom image is successfully created, you will need to backup the associated VHD into another storage account so you can recover from it every time you build environment in Microsoft Azure. To create the VM again from that .VHD use the generalized, sysprep procedure for your images. We will cover this later in the article.

If you have custom VM images, only upload fixed VHD’s (not dynamic, and not .VHDX). You use the Microsoft Azure PowerShell cmdlets to add your custom VM images up into Microsoft Azure by using Add-AzureVHD which automatically converts dynamic VHD’s to fixed VHD’s before uploading them to Microsoft Azure.

Before you upload a custom VM Image to Microsoft Azure, please refer to my earlier article once again Part 18 - Migrating Physical Servers to Virtual Machines so you have a better understanding of how to migrate and create a .VHD file. In the previous article, I cover the Disk2vhd tool and getting the .vhd file ready to either be an image or a disk within Microsoft Azure.

Prerequisites

An Azure subscription - If you don't have one, you can create a free trial account in just a couple of minutes. For details, see Create an Azure account.
Microsoft Azure PowerShell - You have the Microsoft Azure PowerShell module installed. To download the module, see Microsoft Azure Downloads. Below is a tutorial on how to install and configure PowerShell with your Azure Subscription here. The Add-AzureVHD cmdlet, which is part of the Microsoft Azure PowerShell module. We'll use this cmdlet to upload the VHD in the ‘Step-By-Step” later in this article.
A supported Windows operating system is stored in a .vhd file. You will then install a supported Windows Server operating system into that virtual hard disk. Multiple tools exist to create .vhd files. You can use a virtualization solutions such as Hyper-V to create the .vhd file and install the operating system. For instructions, see Install the Hyper-V Role and Configure a Virtual Machine.

Important: Once again the VHDX format is not supported in Microsoft Azure. You can convert the disk to VHD format using Hyper-V Manager or the Convert-VHD cmdlet. A tutorial on this can be found here.

Window Server operating system media. This task requires an .iso file that contains the Windows Server operating system.

Once again, to better understand the differences in terminology around a VM, Image and Disk, I built this table for your review:

A virtual machine runs the operating system that you choose when you create the VM and creates the .vhd file for you. Microsoft Azure stores a virtual machine's operating system in a virtual hard disk format or .vhd file. Hyper-V hosts can use both .vhd and .vhdx.

An image is a .vhd of an operating system that has been prepared for duplication for other like systems so it is required that it be generalized using the System Preparation Tool. This is because it’s going to be imaged to be used on other like systems, so the “Windows Welcome” otherwise known as the “Out of Box Experience or OOBE” has to be rolled back or reset per licensing requirements.

A Disk is used to create new Virtual Machines. If used in a single instance or a “one-off” you do not need to generalize it using Sysprep.

Step 1: Prepare the image to be uploaded

Before the image can be uploaded to Azure, it must be generalized by using the Sysprep command. For more information about using Sysprep, see How to Use Sysprep: An Introduction.

In the virtual machine that you just created, complete the following procedure:

1. Log in to the operating system.

2. Open a Command Prompt window as an Administrator.

Change the directory to %windir%\system32\sysprep, and then run sysprep.exe .

Option 1 - When creating a VHD image for use on Azure from Windows Server 2012 R2 Hyper-V, create it as a Gen-1 VM. Then when you generalize it, you can run “sysprep /generalize /oobe /shutdown /mode:vm” from a command prompt. The “/mode:vm” switch permits new VM’s to be provisioned more quickly on Azure, because it retains the Gen-1 virtual hardware drivers inside the VM instead of forcing hardware to be redetected on each deployment.

My peer Brian Lewis, wrote an article with the details at https://mythoughtsonit.com/2014/06/what-does-the-modevm-switch-do-in-sysprep/

Option 2 - Or you can do the manual Sysprep routine below

3. The System Preparation Tool dialog box appears.

In the System Preparation Tool, select Enter System Out of Box Experience (OOBE) and make sure that Generalize is checked.
In Shutdown Options, select Shutdown Click OK.

The .VHD is now ready to upload to Azure, let’s look at how we do that in the next section.

Step 2: Creating a Storage Account in Azure

A storage account accesses the storage services as it is associated with your Azure subscription. You need a storage account in Azure to upload a .vhd file to Azure that can be used for creating a virtual machine. Use the Azure Management Portal to create a storage account.

1. Sign in to the Azure Management Portal.

2. On the command bar, click New.

3. Click Storage Account then click Quick Create.

Fill out the fields as follows:

4. Under URL, type a subdomain name to use in the URL for the storage account. The entry can contain from 3-24 lowercase letters and numbers.

This name becomes the host name within the URL that is used to address in this case the Blob, but also Queue, or Table resources for the subscription where the .VHD is going to be copied and stored.

5. Choose the location or affinity group for the storage account. By specifying an affinity group, you can co-locate your cloud services in the same data center with your storage for better performance.

6. Decide whether to use geo-replication for the storage account. Geo-replication is turned on by default. I select Local Redundant for my TEST/DEV environment, this is just something to consider.

Note - This option replicates your data to a secondary location, at no cost to you, so that your storage fails over to a secondary location if a major failure occurs that can't be handled in the primary location. The secondary location is assigned automatically, and can't be changed. If legal requirements or organizational policy requires tighter control over the location of your cloud-based storage, you can turn off geo-replication. However, be aware that if you later turn on geo-replication, you will be charged a one-time data transfer fee to replicate your existing data to the secondary location. Storage services without geo-replication are offered.

Creating the Storage Account

The account now appears under Storage Accounts.

Next, we’ll create a Container for your uploaded VHD. Click on Storage account name and click on Containers.

8. Click Create a Container.

9. Type a Name for your container and select Access Policy.

Create the Container

This is where you’re going to copy your new .VHD into this new blob storage area that is being created.

Installing Microsoft Azure PowerShell

We now have to take the steps to install the Microsoft Azure PowerShell Cmdlets- How to: Install Microsoft Azure PowerShell.

You can use Microsoft Azure PowerShell to perform a variety of tasks in Azure, either interactively at a command prompt or automate through scripts.

Azure PowerShell is a module that provides a special set of cmdlets to manage Azure through Microsoft Azure PowerShell. You can use the cmdlets to create, test, deploy, and manage solutions and services delivered through the Azure platform like what we are doing with a Custom VM. In most cases, you can use the cmdlets to perform the same tasks that you can perform through the Azure Management Portal. For example, you can create and configure cloud services, virtual machines, virtual networks, and websites.

So this is the first take is to get this installed on the local, bare metal server.

Go through the Setup, check for the prereqs to be installed.

Launch the Web Platform Installer 5.0

Install Microsoft Azure PowerShell

Continue with the Install.

Make sure and check the prereqs before Installing

Install Microsoft Azure Cross-Platform Command Line Tools

Click I Accept

Step 3: Prepare the connection to Microsoft Azure

Before you can upload a .vhd file, you need to establish a secure connection between your computer and your subscription in Microsoft Azure. You can use the Microsoft Azure Active Directory (Microsoft Account) or the certificate method for doing this.

Use the Microsoft Azure AD method

1. Open the Microsoft Azure PowerShell console, as instructed in How to: Install Microsoft Azure PowerShell.

2. Type the following command: Add-AzureAccount. A window opens up where you can sign into Azure with your Microsoft account.

3. Microsoft Azure authenticates and saves the credential information, type in your Microsoft Account information.

It then validates the Account, if the wrong subscription is associated with the Add-AzureAccount, you can type in Select-AzureSubscription, and it will list them out.

You then type in the subscription name in the SubscriptionNameprompt

You can also use the certificate method for Account Validation

1. Open a Microsoft Azure PowerShellwindow.

2. Type: Get-AzurePublishSettingsFile.

3. A browser window opens and prompts you to download a .publishsettingsfile.

It contains information and a certificate for your Microsoft Azure subscription.

4. Save the .publishsettingsfile.

5. Type: Import-AzurePublishSettingsFile <PathToFile>

Where <PathToFile> is the full path to the .publishsettings file.

For more information on installing and configuring PowerShell, see How to install and configure Microsoft Azure PowerShell

For more information, see Get Started with Microsoft Azure Cmdlets

Step 4: Upload the .vhd file

When you upload the .vhd file, you can place the .vhd file anywhere within your blob storage. In the following command examples, BlobStorageURL is the URL for the storage account that you created in Step 2, YourImagesFolder is the container within blob storage where you want to store your images. VHDName is the label that appears in the Management Portal to identify the virtual hard disk. PathToVHDFile is the full path and name of the .vhd file.

1. From the Microsoft Azure PowerShell window you used in the previous step, type:

Add-AzureVhd -Destination "<BlobStorageURL>/<YourImagesFolder>/<VHDName>.vhd" -LocalFilePath <PathToVHDFile>

Add-Azurevhd –Destination " https://bbvhdazuresa01.blob.core.windows.net/vhds/blainsvr01azure.vhd" -LocalFilePath "D:\AzureUploads\blainsvr01azure.vhd" -NumberofuploaderThreads 5

The .VHD is copied to the Storage Accounting Microsoft Azure

For more information about the Add-AzureVhd cmdlet, see Add-AzureVhd.

Add the Image to Your List of Custom Images

After you upload the .vhd, you add it as an image to the list of custom images associated with your subscription.

1. From the Management Portal, under All Items, click Virtual Machines.

2. Under Virtual Machines, click Images.

3. And then click Create an Image.

4. In Create an image from a VHD, do the following:

Specify name
Specify description
Specify the URL of your VHD click the folder button to launch the below dialog box

- Select the storage account your VHD is in and click Open. This returns you to the Create an image from a VHD window. -

After you return to the Create an image from a VHD window, select the Operating System Family. -

Check I have run Sysprep on the virtual machine associated with this VHD to acknowledge that you generalized the operating system in Step 1, and then click OK.

6. After you complete the previous steps, the new image is listed when you choose the Images tab.

When you create a new virtual machine, you can now use this new image. Choose My Images to show the new image. For instructions, see Create a Virtual Machine Running Windows Server.

1. After you complete the previous steps, the new image is listed when you choose the Images tab.

When you create a new virtual machine, you can now use this new image. Choose My Images to show the new image. For instructions, see Create a Virtual Machine Running Windows Server.

You then build your Custom VM Image from the Gallery under MY IMAGES

You can then do an RDP to the custom VM to manage it.

Using Resource Groups to manage your Microsoft Azure Resources

Resource groups enable you to manage all your resources in an application together. Resource groups are enabled by the new management functionality, Azure Resource Manager. Azure Resource Manager allows you to group multiple resources as a logical group which serves as the lifecycle boundary for every resource contained within it. Typically a group will contain resources related to a specific application. For example, a group may contain a Website resource that hosts your public website, a SQL Database that stores relational data used by the site, and a Storage Account that stores non-relational assets.

An Azure resource groupis a collection of Azure resources (like SQL, VM’s, Website Storage) that are deployed as a unit.

Currently in today’s Azure Portal, managing a resource (a user-managed entity such as a database server, database or website,) requires you to perform operations against one resource at a time. If you had a complex application made up of multiple resources, management of this application became a complex task. In the Microsoft Azure preview portal you can create resource groups to manage all your resources in an application together.

Here is an overview of how to use Resource groups within the Microsoft Azure Preview Portal.

Browsing Resource groups

You can browse all resource groups by clicking the Jumpbar on the left side of your screen. A resource group has a blade that gives you all the information on a particular resource group. The Resource group blade will also give you a unified view of your billing and monitoring information for all the resources in the Resource group.

The summary section shows a visual resource map of all the resources in the resource group, it also shows resources in other resource groups that are linked to the resource group. The resource map also shows the status of each resource.

Clicking on the resource map launches the list view of all the resources on the resource map. This view will list all the resources within a resource group or linked to it. Clicking on these resources will launch their blades.

Creating Resource groups – Done for you!

Whenever a resource is created in the preview portal, it is always created within a resource group. You can choose to create a new resource group or use an existing resource group in the create flow.

An Azure resource is a user-managed Azure entity, such as a database server, database, or web site. An Azure resource groupis a collection of Azure resources that are deployed as a unit.

When you create an application that consists of a few resources working together (e.g. Website + Database) it is always created in its own resource group, so you can manage the lifecycle of all related assets using the resource group. You can add or remove additional resources from the resource group as your application evolves.

You can go to the + and add in additional resources to your Resource Groups. Here I can add in a SQL Server into my Resource Group within my blab01. You see the summary of resources.

Resource groups in the Azure Preview Portal also allow billing charges to be rolled up at a resource group level, so that customers can better review and monitor the overall cloud costs for running an application composed of multiple resources as a unit. In addition, role-based access control can also be assigned at a Resource Group level within the Azure Preview Portal to delegate access to Azure AD users for selected Resource Groups.

Here’s a look at my Resource Group Billing.

Adding resources to Resource groups

You can add resources to a resource group.

Within the Resource Group, you can click on the + and add a resource.

Deleting Resource groups

Caution - Something to keep in mind is when you are deleting a resource group since there might be other resources linked to it. You can see the linked resources in the resource map and take the necessary steps to avoid any unintentional consequences when you delete resource groups. Since resource groups allow you to manage the lifecycle of all the contained resources, deleting a resource group will delete all the resources contained within it. You can also delete individual resources within a resource group.

Using PowerShell to Create Resource Group Templates

For information on the syntax for Azure PowerShell for Resource Groups go to: https://msdn.microsoft.com/en-us/library/dn654594.aspx

Note- The PowerShell cmdlets for Resource Groups currently only permit provisioning of PaaS resource group templates via PowerShell.

Support for IaaS resource group templates in PowerShell is not yet in place for the Preview Portal.

You can create a resource group with just a name and location, and then use the New-AzureResource cmdlet to create resources and add them to the resource group. However, typically, you use a resource group template, which is a JSON-based model of a resource group for a complex cloud-based service, such as a web portal. The template includes parameter (placeholders) for required resources and configurable property values, likes names and sizes. You can find many templates in the Azure template gallery

(Get-AzureResourceGroupGalleryTemplate) and you can create your own templates. Azure resource group templates are text files in JavaScript Object Notation (JSON) format.

The New-AzureResourceGroup cmdlet creates an Azure resource group and its resources and returns an object that represents the resource group.

Use this cmdlet to create a new resource group. To add a deployment to an existing resource group, use the New-AzureResourceGroupDeployment cmdlet. To add a resource to an existing resource group, use the New-AzureResourcecmdlet.

Hope you learned a few things about Custom VM Images and, the other topic, Resource Groups.