Deploying a VM to Windows Azure with Orchestrator leveraging the Service Manager CMDB – Part 1

Hello readers! I’m Tiander Turpijn, I’m a Program Manager within the Customer Architecture & Technology (CAT) team here at Microsoft and I will start contributing to our Building Clouds team blog as of today. I will be focusing on Datacenter and System Center automation, but will also talk about other System Center components like for example Operations Manager.

In this blog post I will show you how you can extend the fantastic Windows Azure Orchestrator runbooks which my fellow team member Charles Joy has posted here: https://blogs.technet.com/b/privatecloud/archive/2013/02/22/automation-orchestrating-windows-azure-solving-the-public-cloud-puzzle-with-system-center-2012-sp1.aspx

In this blog post I will describe how I have created a custom Service Manager management pack and a “just deploy a Windows Azure VM” Orchestrator runbook. This blog post is not intended to fully walk you through all the necessary steps, but to give you guidance and recommendations while creating it. At the end of this blog post you will find a link to download the Service Manager management pack.

If you are using and have tested Charles Joy’s runbooks you know that he is using a text file called “AzureDeploymentDemo.txt” to store all the Windows Azure VM configuration data which will be parsed by Orchestrator to retrieve it. Since I wanted to call the Windows Azure runbooks from the System Center Service Manager self-service portal, I came up with the idea to store the configuration data into the Service Manager’s CMDB. From there I could then have the user pick a Windows Azure VM configuration and decide later on which fields I would like to “override”. So my first objective was to replace the “AzureDeploymentDemo.txt” , I’ll talk later on the challenges with that.

Here’s a summary what we will be doing:

1. Create a new configuration item class in Service Manager
2. Create a custom form
3. Create a Service Manager console view
4. Create a Service and a Request Offering for the self-service portal
5. Create an Orchestrator runbook, which will be called from Service Manager
6. Deploy a Windows Azure VM through the self-service portal

My assumptions:

• You have downloaded and configured Charles Joy’s Windows Azure runbooks, including a Windows Azure subscription
• You have the Orchestrator Service Manager 2012 SP1 integration packs installed and configured
• You have Service Manager 2012 SP1 installed and configured, including the self-service portal
• You have the Service Manager Authoring Tool downloaded, installed and are somewhat familiar how to use it

 

Step 1. Create a new configuration item class in Service Manager

So for the Windows Azure VM configuration to come out of Service Manager we will need to create a new configuration item class which you can do with the System Center Service Manager Authoring Tool which can be downloaded here: https://www.microsoft.com/en-us/download/details.aspx?id=36214

 

Start the Authoring Tool, click “File” and then “New” and let’s create a new management pack and a new configuration item class:

Since I wanted an unique identifier (not a GUID) which I could refer to, I decided to create an “Azure VM Config ID” which I made my key property. I initially wanted to use my Azure VM ConfigName as an unique name (sounds logical doesn’t it?), but quickly found out that if you delete that configuration, you can’t re-use that same name, because it is still in the database with the status of “pending delete”.  So first delete the famous “Property_4” default key property:

If you want your new key property to be filled in and incremented automatically then set these two fields:

This will automatically increment your ID without prompting the user for a value when the configuration form is being filled in.

The next step is to think about which fields you need and would like to have. Your logical starting point I think is to look at the “AzureDeploymentDemo.txt”   text file and get those fields in there which I’ve done to start with:

Since my initial objective was to “just deploy a Windows Azure VM” I initially used only those fields to just do that, you can pretty much create your own property if you see a need for it. My recommendation to you is to really overthink the property you are adding. Is it a configuration property or for example a connection string, which probably should be stored in another place.

 

Step 2. Create a custom form

Now that you have added all the custom properties to your new configuration item class, you can go all fancy with creating a custom form. You can add text labels, dropdown boxes, a logo, different font types and colors, etc.:

Use the “Form Customization Toolbox” to create the form you would like, just start drop and dragging:

And you will end up with something like this:

Notice that I’m using dropdown boxes for fields that are common or fields that you don’t want your users to become “creative” with like VM sizes or the source image you would like them to use. To achieve that you change the property to a “list data type” property:

And then create a custom list:

If you would like to put entries/values in the list you can edit your new management pack with a XML editor or you can add those values later in the Service Manager console. Here’s a snippet view from Visual Studio:

If you are not comfortable with XML editing then import your new management pack into Service Manager, add values to your new list, export your management pack and open the modified management pack in the authoring console.

Note: This would be a good time to start saving different versions of your management pack in case something goes wrong and you need to revert back

 

Step 3. Create a Service Manager console view

Now that you have created your custom configuration class, a custom form, your lists and added those to dropdown boxes onto your form, it is time to finally import your new management pack into Service Manager. After importing your MP you can create a new folder & view to expose your new class and form:

If you create your new view, you need to browse for your new configuration item class:

Then add all the the columns you wish to display.

Before you create your first Windows Azure VM configuration, make sure that all the lists are populated. Go under “Library” and then “Lists” and search for you custom class:

Verify that you have entries:

Now go back to “Configuration Items” node and create your first Windows Azure VM configuration:

 

Step 4. Create a Service and a Request Offering for the self-service portal

Now that we have a Windows Azure VM configuration stored into the CMDB the next step is to make a Request and a Service Offering. Since there are a bunch of blog posts out there on how to do that, I won’t dive into it right here, but one thing to consider is to determine which values you would like to be “overwritten”. You could go ahead and deploy this configuration as is or you could decide which fields you would allow to be set through self-service. To keep it simple and to test drive what I’ve created, I decided to just allow the self-service user to select a Windows Azure configuration and to provide only the Virtual Machine Name. Your Request Offering would then look like this:

Configure your query:

Map your prompt to the “Runbook Automation Activity”:

You might wonder why only map the VM Name and not the selected Azure VM Configuration? Well remember the golden rule when you retrieve Service Manager data from Orchestrator. If your data is in the Service Manager CMDB, you don’t have to map that and don’t put it into the “Initialize Data” activity as a parameter, just use Orchestrator to retrieve that CMDB relationship, that is the beauty of a correlated CMDB. In addition you cannot map a multi-value instance to a string.

 

Step 5. Create an Orchestrator runbook, which will be called from Service Manager

Now the Service Manager side is ready, you can leverage Charles Joy’s comprehensive Windows Azure runbooks in a number of ways. You can create a “master” runbook to retrieve the Service Manager relationships and invoke several other runbooks from there. To quickly test drive what has been created and to “just deploy a Windows Azure VM” you could also create a simple runbook which looks like this:

Don’t get intimidated because of the number of activities. If you watch closely you see that the first 6 activities – after the “Initialize Data” activity – are all about retrieving the Service Manager CMDB relationships. You actually only need 4 Windows Azure activities to deploy a VM, including the creation of a Cloud service and storage. The rest of those are all meant to impress whoever you would like to impress and demonstrate the flexibility and power of System Center.

The Initialize Data activity only contains, besides an identifier, the AzureVMname in my simple test scenario:

I’m using the standard Windows Azure Orchestrator activities which you need to populate with fields coming from the Service Manager CMDB. To do that you just retrieve the relationship one time where the “related class” is your new configuration item class:

Set your filter settings as follow and you’re done:

Now configure all the activities so that all fields come from the data bus “Get Object” activity (called “Get Azure VM Config” in my example)

You will notice that some fields cannot use a data bus value like this one:

Since I’m anticipating for Windows Azure PowerShell usage, I decided to keep those fields in the config, although you can’t use them in the standard Windows Azure activities.

Because I don’t like to store passwords in a configuration - which I think is a bad practice -  I’ve added a “Generate Admin Password” activity so that you can decide how complex your password needs to be:

The downside is that if you don’t have proper notification or registration in place, you will end up with a deployed VM which you cannot RDP into because of the automated password generation, but hey, good security has its price

To have notification in place which not only sends confirmation to the requesting user - in Service Manager terminology called the “affected user” - but also sends the generated password, I’ve added a “Send Email” activity. In here you can add data bus values like AzureVMname and password.

For the email to be sent to the “affected user” - so the user who has requested the VM - you need to use some get relationship activities to get that user (Get SR relationship, Get Affected User Relationship, Get Affected User):

If you want to create a really fancy HTML email with nice formatting and colors, just create your email in Word and save as HTML. Then cut and paste the HTML code in the message area of the “send email” activity and pick your fields where you want Orchestrator data bus values to appear, like in the example below.

Snippet from the send email activity (using HTML for fancy formatting):

Let’s do one more fancy thing and that is to add the RDP file as an attachment to the email so that your user just has to open the RDP file to connect to the new Windows Azure VM, isn’t it cool that we have a standard Windows Azure cascading activity for that?:

Add it to the email and we’re done:

 

Step 6. Deploy a Windows Azure VM through the self-service portal

Now that we’re all set up, it’s time to test drive!

Let’s start with the Service Manager self-service portal (assuming that you have at least 1 Azure VM Config entry):

You can have your users look at the “Gold” configuration by clicking on it:

Let’s submit the Service Request:

Orchestrator runbook has successfully run:

Checking the Windows Azure portal, looks good:

Got a nice email confirmation (notice the attachment, it’s the RDP file):

Open the RDP file:

Logged on to our new Windows Azure VM!:

 

Where can I get the Service Manager sample management pack?

To save you time and effort in building your own custom class and custom form I have posted the Service Manager management pack on the Technet Gallery, you can find it here: https://gallery.technet.microsoft.com/Windows-Azure-VM-Config-878f0c39

The management pack isn’t sealed so that you can open it from the Service Manager Authoring Tool and modify what you would like. If you are going to test it further it is recommended to seal the management pack so that you can reference it in other management packs. Sealing your management pack is really easy to do from the Authoring Tool:

 

Closing thoughts, conclusion and what is coming next:

The primary objective of this blog post was to replace the text file as used in Charles Joy’s his runbooks - which holds all the Windows Azure VM configuration data – with data coming from and stored in the Service Manager CMDB.

In the process of testing and taking this a step further I found that having one single Windows Azure VM configuration introduces a number of limitations, for example:

• You can only use this for deploying one single VM
• What if you would like to deploy an additional VM to the same Cloud Service, running under the same Storage Container?
• What if you only wanted to delete the VM and re-deploy a VM with the same Cloud Service and Storage configuration
• ….

Conversations with my fellow team Program Manager Anders Ravnholt have led to a couple of additional thoughts. We could do so much more and with more flexibility if we separate out the Windows Azure Service and Storage configuration from the VM instance configuration, hmm….interesting isn’t it? Thoughts about capacity - how many VM’s do I have running on which Cloud, either Public or Private? - How about charge-back? Etc.?

So please stay tuned for the next blog post where I show you how to separate the Windows Azure Service and Storage from the VM instance configuration and what you can do with that.

I hope you found this blog post useful, happy testing!

Tiander Turpijn.