Azure Image Management Automation


 

Hello, in this blog post, I will talk about the Azure Image Management Automation which is a set Azure Automation Runbooks that allows us to alleviate some of the overhead work that comes with the adoption of custom “golden” images built on-premises and used in Azure.

This solution is a team effort between John Knightly (which is also a PFE) and me, where he could propose an initial solution to his customer and after some discussions between us we could design, develop  and test this solution.

Overview

To give a little perspective of how does this solution may help you, lets think about the manual process of having to build a custom golden image (steps 1-3 will exist anyways):

  1. On Hyper-v or whatever hypervisor solution you use, install the OS from an ISO
  2. Perform your customizations
  3. Run SYSPREP (for Windows) or waagent -deprovision+user (for Linux)
  4. Upload the VHD to a Storage Account by using Azure Storage Explorer, AzCopy, Add-AzureRmVHD, etc.
  5. Create a managed Image of this VHD in order to be able to easily deploy VMs from that golden image

 

These steps are actually very simple but that simplicity stops when you need to make that image available to a second region, in this above example I uploaded the VHD in West US region and into a single subscription, now, what if you need to start deploying VMs from the same golden image throughout one more region? Simple, copy the VHD from the Storage Account in West US to West US 2 and then create a new managed image in that region. Why is that? Because managed images can be only created from VHDs located in the same region. Now, let’s add more, what if you have a DEV, QA and PROD subscriptions? Well, multiply the two steps by 3, giving you 6 operations. Expanding a little bit more, add a Windows and Linux golden image. This math will go to 12 operations you need to manually execute and monitor. Do that each month if you have a monthly golden image release cycle. Already too much work for a small shop correct?

To close this thought, imagine you’re a big global company with lots of business units and each one with its own requirements on Azure, leading you to have 300 subscriptions, be present in at least 10 Azure regions and need to maintain at least two golden images, one for Windows and one for Linux? Well, that will result in 6,000 manual operations a month. This would result in lots o man/hours of work and be error prone.

This solution helps you to simplify and automate this process, what you need to do is to continue producing your golden images and just use a single script to upload the VHD and wait the images being created in each subscription and region.

 

It uses the following Azure components:

  1. Azure Active Directory
  2. Azure Storage Accounts
  3. Azure Storage Tables
  4. Azure Storage Queues
  5. Azure Automation

 

We are also relying on two PowerShell modules I built in order to interact with Storage Tables and Storage Queues. It also depends on a module created specifically for this solution to avoid code duplication as much as possible.

 

The following diagram shows how the process works, from uploading the VHD to producing the managed Image.

 

image

 

Before I continue to more details, let me define some terms:

  • Tier 0 Storage Account – Storage account that contains the configuration table, the queues for VHD copy process and Managed Image creation process. Finally it is the initial point where the VHD is uploaded first.
  • Tier 1 Blobs – Multiple copies of the original VHD are made inside of the Tier 0 Storage Account to allow multiple parallel copies (this is defined during setup).
  • Tier 0 Subscription – Subscription where all automation components are created.
  • Tier 2 Subscriptions – Subscriptions that holds tier 2 Storage Accounts and will get the Managed Image created.
  • Tier 2 Storage Accounts – Final destination of the VHDs, they can be in same or different subscription, different regions, etc.

 

Step by step explanation:

  1. After building your VHD, use UploadVHD.ps1 script to upload it to a tier 0 storage account
  2. UploadVHD.ps1 script will upload the VHD to Tier 0 Storage Account and will trigger Start-ImageManagementTier1Distribution runbook
  3. Start-ImageManagementTier1Distribution will create multiple copies of the VHD in order to allow parallel copies (when using Start-AzureStorageBlobCopy, the source blob is locked until the copy is in progress, meaning that no other copy operation can start)
  4. After all copies are done, a message is placed in the copy-process-queue stating that there is a VHD to distribute to Tier 2 Storage Accounts (with other information that will be consumed later)
  5. Runbook Start-ImageManagementTier2Distribution is scheduled to run every hour (minimum value) and check the copy-process-queue, if there is a message there, it will trigger one runbook called Start-ImageManagementVhdCopyTier2 that resides in a different Automation Account per destination Tier 2 Storage Account. Note that this is in another Automation Account due to the 200 runbook jobs running at the same time, for a complete list of limitations, please refer to https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits#automation-limits)
  6. After each individual runbooks completes its execution, a message per copied VHD is placed in the image-creation-process-queue
  7. Another scheduled runbook called Start-ImageManagementImageCreation will monitor that queue (hourly again) and trigger one New-ImageManagementImage runbook (located in another Azure Automation Account due to same reasons already explained, it also can be in multiple Automation Accounts, all depends on how you did the initial setup) per VHD and perform the image creation.
  8. The end result is that the managed images will start showing up at the locations you defined.

 

Setup

 

Software/Accounts Requirements

In order to be able to install this solution you will be required to:

  1. Execute in a computer with PowerShell 5.0 or that has PS Get installed, the installation/source code section of this blog post describes how to do it.
  2. Install the following modules from PowerShell Gallery

    Install-Module MSOnline
    Install-Module AzureRmImageManagement
  3. Execute in an elevated PowerShell command prompt Window
  4. Be using an Azure account that is a Global Admin in Azure Active Directory that manages Identities for all involved subscriptions
  5. This account also must be Owner of each Azure Subscription involved

Download

This solution can be downloaded from my GitHub repository.

 

SetupInfo.json file

In order to provide an easy to use solution, I provided a setup script that will automate the installation process for you. But just before we go into this process, we need to review the SetupInfo.json as follows:

 

{
    "general":
    {
        "tenantName":"yourtenant.onmicrosoft.com",
        "copyProcessQueueName":"copy-process-queue",
        "imageCreationQueueName":"image-creation-process-queue"
    },
    "requiredModulesToInstall":[
        "AzureRmStorageTable",
        "AzureRmStorageQueue",
        "MSOnline",
        "AzureRmImageManagement"
    ],                  
    "storage":
    {
        "tier0StorageAccount":
        {
            "storageAccountPrefix":"pmcstorage77",
            "resourceGroup":"imageprocess-rg",
            "location":"westus2",
            "subscriptionId":"af86d43c-d9b4-4b18-a1a4-1db9830c015a",
            "container":"vhd",
            "modulesContainer":"modules",
            "configurationTableName":"imageManagementConfiguration",
            "tier1Copies":10
        },
        "tier2StorageAccounts":
        [
            {
                "storageAccountPrefix":"pmcstorage77sub1",
                "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
                "location":"^$config.storage.tier0StorageAccount.location",
                "container":"^$config.storage.tier0StorageAccount.container",
                "subscriptionId":"^$config.storage.tier0StorageAccount.subscriptionId"
            },
            {
                "storageAccountPrefix":"pmcstorage77sub2",
                "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
                "location":"^$config.storage.tier0StorageAccount.location",
                "container":"^$config.storage.tier0StorageAccount.container",
                "subscriptionId":"af86d43c-d9b4-4b18-a1a4-1db9830c015a"
            }
        ]
    },
    "automationAccount":
    {
        "subscriptionId":"^$config.storage.tier0StorageAccount.subscriptionId",
        "applicationDisplayNameSuffix":"TestAutomationSP",
        "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
        "location":"southcentralus",
        "automationAccountNameSuffix":"ImageManagementAutomation",
        "workerAutomationAccountsCount":2,
        "availableDedicatedCopyJobs":150,
        "maxDedicatedCopyJobs":150,
        "availableDedicatedImageCreationJobs":150,
        "maxDedicatedImageCreationJobs":150,
        "connectionName":"AzureRunAsConnection",
        "runbooks":
        {
            "mainAutomationAccount":
            [
                {
                    "name":"Update-ModulesInAutomationToLatestVersion",
                    "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":true,
                    "parameters":[],
                    "requiredModules":[]
                },
                {
                    "name":"Start-ImageManagementTier1Distribution",
                    "scriptPath":"..\\Runbooks\\Start-ImageManagementTier1Distribution.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":false,
                    "parameters":[],
                    "requiredModules":[
                        "AzureRmStorageTable",
                        "AzureRmStorageQueue",
                        "MSOnline",
                        "AzureRmImageManagement"
                    ]
                },
                {
                    "name":"Start-ImageManagementTier2Distribution",
                    "scriptPath":"..\\Runbooks\\Start-ImageManagementTier2Distribution.ps1",
                    "scheduleName":"HourlySchedule",
                    "scheduleHourInterval":1,
                    "executeBeforeMoveForward":false,
                    "parameters":[
                        {
                            "key":"ConfigStorageAccountResourceGroupName",
                            "value":"^$config.storage.tier0StorageAccount.resourceGroup"
                        },
                        {
                            "key":"ConfigStorageAccountName",
                            "value":"^[string]::Format(\"{0}tier0\",$config.storage.tier0StorageAccount.storageAccountPrefix)"
                        },
                        {
                            "key":"ConfigurationTableName",
                            "value":"^$config.storage.tier0StorageAccount.configurationTableName"
                        },
                        {
                            "key":"Tier0SubscriptionId",
                            "value":"^$config.storage.tier0StorageAccount.subscriptionId"
                        },
                        {
                            "key":"ConnectionName",
                            "value":"^$config.automationAccount.connectionName"
                        }
                    ],
                    "requiredModules":[
                        "AzureRmStorageTable",
                        "AzureRmStorageQueue",
                        "MSOnline",
                        "AzureRmImageManagement"
                    ]                    
                },
                {
                    "name":"Start-ImageManagementImageCreation",
                    "scriptPath":"..\\Runbooks\\Start-ImageManagementImageCreation.ps1",
                    "scheduleName":"HourlySchedule",
                    "scheduleHourInterval":1,
                    "executeBeforeMoveForward":false,
                    "parameters":[
                        {
                            "key":"ConfigStorageAccountResourceGroupName",
                            "value":"^$config.storage.tier0StorageAccount.resourceGroup"
                        },
                        {
                            "key":"ConfigStorageAccountName",
                            "value":"^[string]::Format(\"{0}tier0\",$config.storage.tier0StorageAccount.storageAccountPrefix)"
                        },
                        {
                            "key":"ConfigurationTableName",
                            "value":"^$config.storage.tier0StorageAccount.configurationTableName"
                        },
                        {
                            "key":"Tier0SubscriptionId",
                            "value":"^$config.storage.tier0StorageAccount.subscriptionId"
                        },
                        {
                            "key":"ConnectionName",
                            "value":"^$config.automationAccount.connectionName"
                        }
                    ],
                    "requiredModules":[
                        "AzureRmStorageTable",
                        "AzureRmStorageQueue",
                        "MSOnline",
                        "AzureRmImageManagement"
                    ]                    
                }
            ],
            "copyProcessAutomationAccount":
            [
                {
                    "name":"Update-ModulesInAutomationToLatestVersion",
                    "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":true,
                    "parameters":[],
                    "requiredModules":[]
                },                
                {
                    "name":"Start-ImageManagementVhdCopyTier2",
                    "scriptPath":"..\\Runbooks\\Start-ImageManagementVhdCopyTier2.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":false,
                    "parameters":[],
                    "requiredModules":[
                        "AzureRmStorageTable",
                        "AzureRmStorageQueue",
                        "MSOnline",
                        "AzureRmImageManagement"
                    ]                    
                }
            ],
            "imageCreationProcessAutomationAccount":
            [
                {
                    "name":"Update-ModulesInAutomationToLatestVersion",
                    "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":true,
                    "parameters":[],
                    "requiredModules":[]
                },
                {
                    "name":"New-ImageManagementImage",
                    "scriptPath":"..\\Runbooks\\New-ImageManagementImage.ps1",
                    "scheduleName":null,
                    "scheduleHourInterval":null,
                    "executeBeforeMoveForward":false,
                    "parameters":[],
                    "requiredModules":[
                        "AzureRmStorageTable",
                        "AzureRmStorageQueue",
                        "MSOnline",
                        "AzureRmImageManagement"
                    ]                    
                }
            ]
        }
    }
}

 

This JSON file contains the information needed by the setup script to perform its installation, it is basically split into four main sections as follows:

  • general
  • requiredModulesToInstall
  • storage
  • automationAccount

 

Following items describes in detail all sections and what is required to be changed in order to set this up in your own environment.

 

General section

This section mainly requires the Azure Active Directory tenant name and defines the queue names.

Element Name Description Modification Required?
tenantName That’s the Azure Active Directory tenant name. e.g. test.onmicrosoft.com Yes
copyProcessQueueName Name of the queue monitored by the copy process No
imageCreationQueueName Name of the queue monitored by the image creation process No

 

Section example

"general":
{
    "tenantName":"tenantname.onmicrosoft.com",
    "copyProcessQueueName":"copy-process-queue",
    "imageCreationQueueName":"image-creation-process-queue"
}

 

RequiredModulesToInstall section

This sections needs to remain unchanged unless you want to have additional modules installed in your Automation Account as soon as it gets created, This is consumed by the setup script to download the required module from PowerShell Gallery and make it available in the Tier 0 storage account for later Azure Automation Account deploment.

This is a simple array with the module names that you want to install.

Section example

"requiredModulesToInstall":[
    "AzureRmStorageTable",
    "AzureRmStorageQueue",
    "MSOnline",
    "AzureRmImageManagement"
]

 

Storage section

This section defines the Tier 0 Storage Account information, number of Tier 1 blob copies and other important information, it also defines each Tier 2 Storage Account which is each individual Storage Accounts that will receive a copy of the VHD, can be in the same or different region/subscription.

This section is split between tier0StorageAccount and tier2StorageAccount as follows:

tier0StorageAccount subsection

Defines information about Tier 0 Storage Account. This SA is the one that contains the configuration table, queues and the blobs to be copied.

Element Name Description Modification Required?
storageAccountPrefix This string is used to compose the storage account name, during the setup process the prefix have the string tier0 appended. With that in mind your prefix string cannot be greater than 19 characters. Yes
resourceGroup Name of the resource group that you want this storage account to be created on. Yes
location A valid Azure location, to obtain the Azure location list you can execute Get-AzureRmLocation to get a list of all locations, use location value.
Yes

subscriptionId

This is guid of the subscription that will have the Tier 0 Storage Account created on. Use Get-AzureRmSubscription cmdlet to list all subscriptions you have access. Yes

container

This is the name of the container that will have the VHD copied to plus the tier 1 copies. Default is “goldenvhds”. No

modulesContainer

This is a container at the storage account that will get all necessary modules uploaded to, later on, the setup process will use this as the source for the Azure Automation modules setup. No

configurationTableName

This is the table that has all configurations used by this solution, manual edit of this table can be done later if you need to change something. Defaults to imageManagementConfiguration. No
tier1Copies This is the number of copies inside of Tier 0 Storage Account the VHD blob will have, this number will help you to have more copies done in parallel, so the higher the number, the higher the number of concurrent copies. This dictates how fast step 4 described earlier will be but will put some strand on step 3 since will will have multiple copy operations waiting for the source blob to complete the previous copy so a good number will be based on your own environment. This defaults to 10 copies. Yes

 

tier2StorageAccounts subsection

This is an array that defines information about each Tier 2 Storage Account. This SA is the one that will receive the final VHD blob and will be used by the managed image creation process.

Element Name Description Modification Required?
storageAccountPrefix This string is used to compose the storage account name, during the setup process the prefix have the string tier2 appended. With that in mind your prefix string cannot be greater than 19 characters. Yes
resourceGroup Name of the resource group that you want this storage account to be created on. Yes
location A valid Azure location, to obtain the Azure location list you can execute Get-AzureRmLocation to get a list of all locations, use location value.
Yes

subscriptionId

This is guid of the subscription that will have the Tier 2 Storage Account created on. Use Get-AzureRmSubscription cmdlet to list all subscriptions you have access. Yes

container

This is the name of the container that will have the VHD copied to plus the tier 1 copies. Default is “goldenvhds”. No

 

You will have as many Tier 2 Storage Accounts listed in this section as you need, they need to be added as an extra element of this array. Notice that if in the future you need to expand your solution to have more/new Tier 2 Storage Accounts, make sure you use the same JSON file with the additions and run the setup script again. The script will create the new storage accounts or any other extra items you place here, like new number of Automation Accounts as you will see further in this article.

 

Section example

"storage":
{
    "tier0StorageAccount":
    {
        "storageAccountPrefix":"pmcstorage77",
        "resourceGroup":"imageprocess-rg",
        "location":"westus2",
        "subscriptionId":"af86d43c-d9b4-4b18-a1a4-1db9830c015a",
        "container":"goldenvhds",
        "modulesContainer":"modules",
        "configurationTableName":"imageManagementConfiguration",
        "tier1Copies":10
    },
    "tier2StorageAccounts":
    [
        {
            "storageAccountPrefix":"pmcstorage77sub1",
            "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
            "location":"^$config.storage.tier0StorageAccount.location",
            "container":"^$config.storage.tier0StorageAccount.container",
            "subscriptionId":"^$config.storage.tier0StorageAccount.subscriptionId"
        },
        {
            "storageAccountPrefix":"pmcstorage77sub2",
            "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
            "location":"^$config.storage.tier0StorageAccount.location",
            "container":"^$config.storage.tier0StorageAccount.container",
            "subscriptionId":"c0d433f7-b5e8-441a-91b5-5db70c69d7b8"
        }
    ]
}

 

You will notice in the sample section that we used a different notation for some of the values, We are adding the ^ sign in the beginning of the string, which means that the setup script will execute a PowerShell expression evaluation of the content of the string. In this case what we are referencing is a value inside of the same JSON file. $config is the object we use to load the content of this file, so you can use the paths that we are showing and reference existing values throughout the file so if you have values that repeats, you don’t need to keep copying all the time, but this is complete optional and if you think that adding the actual values will be more educational for documentation purposes feel free to use the actual values. Intention here is just to offer a way to avoid repeating content.

 

Automation Account section

This section defines your Automation Account settings, we will have here some general settings for the Automation Accounts plus specific sections for three Automation Account types (this is solution wise and does not mean that Azure contains different types of Automation Accounts) as follows:

  • Main Automation Account, this is the one that contains the Start-ImageManagementImageCreation, Start-ImageManagementTier1Distribution and Start-ImageManagementTier2Distribution runbooks.
  • Copy Process Automation Account(s), one or more automation accounts to execute the tier 2 copy process, the number of this type of Automation Accounts will depend on the final number of tier 2 storage accounts you have and if you’re performing one or more uploads at the same time, as it was previously explained, a single Automation Account can execute a maximum of 200 runbook jobs so if you have, lets say 300 subscriptions, you will need at least 2 copy dedicated Automation Accounts in order to be as parallel as possible.
  • Image Creation Process Automation(s), same case of the previous description, this type is dedicated to create the Managed Images.

 

This section contains the following items:

Element Name Description Modification Required?
automationAccountPrefix This string is used to compose the Automation Account name, this name will be used as the main Automation Account, whereas copy Automation accounts will have this prefix appended with –Copy999 where 999 is an incremental counter and image creation Automation Accounts will have –Img999 appended in the same way. Since these account names cannot be greater than 50, this prefix cannot be more than 42 characters. Yes
resourceGroup Name of the resource group that you want the Automation Accounts to be created on. Yes
location A valid Azure location, to obtain the Azure location list you can execute Get-AzureRmLocation to get a list of all locations, use location value.
Yes

subscriptionId

This is guid of the subscription that will have the Automation Accounts created on. Use Get-AzureRmSubscription cmdlet to list all subscriptions you have access. Yes

applicationDisplayNamePrefix

This is the prefix that will be used to create the service principals in Azure Active Directory, these names will end up being with <prefix>-Copy999 and <prefix>-Img999 the same way the Automation Account Prefix. Yes

workerAutomationAccountsCount

This element will dictate how many Automation Accounts will be created for Copy and Image Creation processes, by default it will create two of them, allowing this solution to serve up to 300 subscriptions with one tier 2 storage account and in one location at the same time, increase the number if you want to have more parallel distributions. For example, if we have 100 subscriptions with 3 locations each and maintain one Linux and one Windows image we will end up with (100x3x2) = 600 copy and image creation operations, and in this case we can divide this number by 150 runbook executions and we will have our new number which will be 4.
You can start with a lower number, like the default and increase it to as much as needed and execute the setup gain, setup will add the extra new automation accounts. Note that if you decide to decrease the number this will not remove the extra Automation Accounts, therefore nothing will happen in that setup execution.
Yes (depending of how may copy and image creation operations needed for 100% parallel operations)

availableDedicatedCopyJobs

Number of copy jobs available to be executed, this number decreases and increases as copy jobs gets created or completed. No

maxDedicatedCopyJobs

Maximum number of copy jobs, this number is a control number, used in a way that the solution knows the maximum and perform corrections if needed. No

availableDedicatedImageCreationJobs

Similar case of availableDedicatedCopyJobs setting but dedicated to image creation jobs. No

maxDedicatedImageCreationJobs

Similar case of maxDedicatedCopyJobssetting but dedicated to image creation jobs. No

connectionName

This is the name of the connection to be created (runas account) during setup. No

 

 

Runbooks subsection

This section defines all necessary runbooks to setup in each Automation Account type.

It is split between mainAutomationAccount, copyProcessAutomationAccount and imageCreationProcessAutomationAccount and for the sake of trying to keep this post as small as possible (which is being hard to Smile) I’m describing one element that is used to define a runbook in any of the sections.

 

Element Name Description Modification Required?
name Runbook Name. No
scriptPath Path to the script to be imported into the PowerShell runbook. This path can be a http path as well, when you download the script directly from GitHub raw path for example. No
scheduleName Name of the schedule to create and assign to the runbook, the type is hourly schedule only.
No

executeBeforeMoveForward

Forces the execution of the runbook as soon as it gets imported and wait for it to finish. In our case we add the update runbook from GitHub and execute it in order to update all existing modules in the Automation Account since they are outdated at the moment of account creation, if we don’t execute it all other runbook imports fails. No

parameters

This element define an array of key and value pairs, this defines all parameters that the runbook needs when scheduling its execution. For example:

image
No

requiredModules

An array of modules to import in this automation account if not imported yet. The module list item needs to come from the requiredModulesToInstall section. No

 

Section example

"automationAccount":
{
    "subscriptionId":"^$config.storage.tier0StorageAccount.subscriptionId",
    "applicationDisplayNamePrefix":"TestAutomationSP",
    "resourceGroup":"^$config.storage.tier0StorageAccount.resourceGroup",
    "location":"southcentralus",
    "automationAccountNamePrefix":"ImageManagementAutomation",
    "workerAutomationAccountsCount":2,
    "availableDedicatedCopyJobs":150,
    "maxDedicatedCopyJobs":150,
    "availableDedicatedImageCreationJobs":150,
    "maxDedicatedImageCreationJobs":150,
    "connectionName":"AzureRunAsConnection",
    "runbooks":
    {
        "mainAutomationAccount":
        [
            {
                "name":"Update-ModulesInAutomationToLatestVersion",
                "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":true,
                "parameters":[],
                "requiredModules":[]
            },
            {
                "name":"Start-ImageManagementTier1Distribution",
                "scriptPath":"..\\Runbooks\\Start-ImageManagementTier1Distribution.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":false,
                "parameters":[],
                "requiredModules":[
                    "AzureRmStorageTable",
                    "AzureRmStorageQueue",
                    "MSOnline",
                    "AzureRmImageManagement"
                ]
            },
            {
                "name":"Start-ImageManagementTier2Distribution",
                "scriptPath":"..\\Runbooks\\Start-ImageManagementTier2Distribution.ps1",
                "scheduleName":"HourlySchedule",
                "scheduleHourInterval":1,
                "executeBeforeMoveForward":false,
                "parameters":[
                    {
                        "key":"ConfigStorageAccountResourceGroupName",
                        "value":"^$config.storage.tier0StorageAccount.resourceGroup"
                    },
                    {
                        "key":"ConfigStorageAccountName",
                        "value":"^[string]::Format(\"{0}tier0\",$config.storage.tier0StorageAccount.storageAccountPrefix)"
                    },
                    {
                        "key":"ConfigurationTableName",
                        "value":"^$config.storage.tier0StorageAccount.configurationTableName"
                    },
                    {
                        "key":"Tier0SubscriptionId",
                        "value":"^$config.storage.tier0StorageAccount.subscriptionId"
                    },
                    {
                        "key":"ConnectionName",
                        "value":"^$config.automationAccount.connectionName"
                    }
                ],
                "requiredModules":[
                    "AzureRmStorageTable",
                    "AzureRmStorageQueue",
                    "MSOnline",
                    "AzureRmImageManagement"
                ]                    
            },
            {
                "name":"Start-ImageManagementImageCreation",
                "scriptPath":"..\\Runbooks\\Start-ImageManagementImageCreation.ps1",
                "scheduleName":"HourlySchedule",
                "scheduleHourInterval":1,
                "executeBeforeMoveForward":false,
                "parameters":[
                    {
                        "key":"ConfigStorageAccountResourceGroupName",
                        "value":"^$config.storage.tier0StorageAccount.resourceGroup"
                    },
                    {
                        "key":"ConfigStorageAccountName",
                        "value":"^[string]::Format(\"{0}tier0\",$config.storage.tier0StorageAccount.storageAccountPrefix)"
                    },
                    {
                        "key":"ConfigurationTableName",
                        "value":"^$config.storage.tier0StorageAccount.configurationTableName"
                    },
                    {
                        "key":"Tier0SubscriptionId",
                        "value":"^$config.storage.tier0StorageAccount.subscriptionId"
                    },
                    {
                        "key":"ConnectionName",
                        "value":"^$config.automationAccount.connectionName"
                    }
                ],
                "requiredModules":[
                    "AzureRmStorageTable",
                    "AzureRmStorageQueue",
                    "MSOnline",
                    "AzureRmImageManagement"
                ]                    
            }
        ],
        "copyProcessAutomationAccount":
        [
            {
                "name":"Update-ModulesInAutomationToLatestVersion",
                "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":true,
                "parameters":[],
                "requiredModules":[]
            },                
            {
                "name":"Start-ImageManagementVhdCopyTier2",
                "scriptPath":"..\\Runbooks\\Start-ImageManagementVhdCopyTier2.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":false,
                "parameters":[],
                "requiredModules":[
                    "AzureRmStorageTable",
                    "AzureRmStorageQueue",
                    "MSOnline",
                    "AzureRmImageManagement"
                ]                    
            }
        ],
        "imageCreationProcessAutomationAccount":
        [
            {
                "name":"Update-ModulesInAutomationToLatestVersion",
                "scriptPath":"https://raw.githubusercontent.com/azureautomation/runbooks/master/Utility/ARM/Update-ModulesInAutomationToLatestVersion.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":true,
                "parameters":[],
                "requiredModules":[]
            },
            {
                "name":"New-ImageManagementImage",
                "scriptPath":"..\\Runbooks\\New-ImageManagementImage.ps1",
                "scheduleName":null,
                "scheduleHourInterval":null,
                "executeBeforeMoveForward":false,
                "parameters":[],
                "requiredModules":[
                    "AzureRmStorageTable",
                    "AzureRmStorageQueue",
                    "MSOnline",
                    "AzureRmImageManagement"
                ]                    
            }
        ]
    }
}
}

 

This section is highly sensitive to what runbooks to install, order and modules so if changing it be sure to don’t change the already existing items.

 

Installing the solution

After you review and make the necessary changes to the SetupInfo.json file installation is simple:

  1. Make sure you cleared the requirements described earlier
  2. If not already opened, open an elevated PowerShell window
  3. Login to Azure

    Add-AzureRmAccount


  4. Change folder to <place where you extracted the downloaded solution>\Scripts from GitHub.
  5. Execute

    .\setup.ps1


    if you use a different filename for your JSON configuration file, make sure you use the parameter configFile passing the full path of your file.

    .\setup.ps1 -configFile c:\temp\myconfigfile.json


  6. Wait until the process finishes.

 

Notes

The Automation Accounts created by default are the basic tier in order to overcome the 500 minutes/mo limitation of the free tier. If you need to change it, make sure you change the code where New-AzureRmImgMgmtAutomationAccount cmdlet is called and remove –basicTier (be aware of the ` symbol that needs to be removed in the right above line. Exposing this as parameter in the setup script is in the roadmap.

 

End result example

The following screenshot shows the end result of the setup process in my tier 0 subscription.

 

image

 

At my Tier 2 subscription we will have:

image

 

Configuration table

image

Uploading your VHDs using this process

After installing, the process is very straight-forward as follows:

  1. Open a PowerShell command prompt window
  2. Authenticate on Azure

    Add-AzureRmAccount


  3. Change folder to the <place where you extracted the solution>\Script
  4. Execute the UploadVHD.ps1 script

    $tier0SubscriptionId = "1a1810a5-737e-4182-aa17-15788d0723d6"
    $configStorageAccountResourceGroupName = "myresourcegroup"
    $configStorageAccountName = "mytier0storageaccount"
    $imageName = "WindowsServer2016-20170813"
    $imageResourceGroupName = "Images-RG"
    $vhdFullPath = "c:\temp\WindowsServer2016-GoldenImage-20170813.vhd"
     
    .\UploadVHD.ps1 -Tier0SubscriptionId $Tier0SubscriptionId `
            -ConfigStorageAccountResourceGroupName $ConfigStorageAccountResourceGroupName `
            -ConfigStorageAccountName $ConfigStorageAccountName `
            -ImageName $imageName `
            -VhdFullPath $vhdFullPath `
            -OsType "Windows" `
            -ImageResourceGroupName $imageResourceGroupName 


    Parameters accepted by UploadVHD.ps1 script:

    Parameter Name

    Description

    ConfigStorageAccountResourceGroupName

    Resource Group name where the Azure Storage Account that contains the system configuration tables.

    ConfigStorageAccountName

    Name of the Storage Account that contains the system configuration tables.

    ConfigurationTableName

    Name of the configuration table, default to ImageManagementConfiguration, which is the preferred name. Optional parameter.

    VhdFullPath

    Full path (path + file name) of the VHD to be uploaded to the tier 0 storage account

    ImageName

    Name of the Image that will be generated after the VHD is copied to all subscriptions

    ImageResourceGroupName

    Name of the resource group where the image will be created.

    UploaderThreadNumber

    umber of threads used by Add-AzureRmVhd cmdlet to speed up the upload process, if not provided, it will default to 10 threads.

    Overwrite

    Indicates if file must be overwriten or not, default to no if switch is not provided

    Tier0SubscriptionId

    Tier 0 subscription Id, this is the subscription that contains all runbooks, config storage account and receives the VHD upload from on-premises

    OsType

    VHD's Operating System type, valid ones are Windows and Linux.


After this script is done uploading the blob it will trigger the distribution process (as described in the overview section) and completion time will depend on several factors like image size, number of configured copies of the blob at tier 0 storage account, number of tier 2 subscriptions, storage accounts regions.

 

This is the end result of my sample:

Subscription 1

image

Subscription 2

image

 

Roadmap

This roadmap is not a commitment but is more of a guidance on what could (or not) be implemented in the next few months:

  • Improve setup script deployment time by executing the Automation Accounts deployment in parallel
  • Implement the option to define if Automation Accounts are create as Basic or Free pricing tiers
  • Implement a centralized log with a log table and access the items per Activity ID (each upload execution will have a different Activity ID)
  • Create a simple cmdlet that will give you the global status of the image creation process

 

That’s it for this blog post and I hope that this can help.

 

Regards

Paulo


Comments (1)

  1. Great work Paulo, I appreciate the name drop, but this is your code! 🙂

Skip to main content