Troubleshooting Windows Azure Pack, SPF & VMM
As a follow-up from the troubleshooting Installation of Windows Azure Pack it’s time to look at the next item on our list being troubleshooting Service Provider Foundation (SPF) & Virtual Machine Manager (VMM). In this blog post I will describe how you can:
- Verify integration between WAP, SPF and VMM is working correctly
- Verify SPF is working correctly end to end
As SPF is the foundation between WAP and System Center, we want to make sure this is configured correctly. If SPF and the System Center back-end is not working as expected, there is really no need to troubleshoot Windows Azure Pack as SPF & System Center issues will just bubble up and cause trouble in WAP.
Let’s start by looking at how we can verify all the components are configured correctly.
How to verify integration between WAP, SPF and VMM is working correctly
As WAP IaaS is made up from different layers (WAP, SPF and VMM), in this section I will give some guidance on how you can verify step by step if things are correctly configured all the way from VMM to WAP.
At the end of the day it comes down to ensuring that the right credentials are used in the right areas throughout the different layers.
We want to look into the following areas to verify they are working as expected. In particular, you should verify that:
- The right user is added to VMM administrators used by SPF.
- SPF server can connect to VMM Using PowerShell.
- SPF Web Service is running under the right user credentials.
- The right user for accessing SPF Web services is configured.
- WAP VM Clouds is registered for SPF using the right user.
Now let me start by describing my environment so there is a common reference:
SPF Service account
SPF local account
SPF Web Service
Pre-requisites before starting the troubleshooting
- Virtual Machine Manager 2012 R2 is running and configured to manage at a minimum one Hyper-V host.
- SPF is running and configured and have VMM 2012 R2 console with PowerShell installed.
- WAP is installed and configured.
Verify that the right user is added to VMM administrators used by SPF.
1. Logon to the VMM Server (VMM01.fabrikam.com) as a member of VMM Admins Group.
2. Start VMM 2012 R2 console.
3. Select Settings and expand Security > User Roles
4. Open the Administrator User Role and select Members from the left menu.
5. Under Members verify that your SPF Service account is listed. e.g fabrikam\!sc.
6. Close VMM Console
Verify that the SPF server can connect to VMM using PowerShell
As SPF uses PowerShell to execute commands against VMM we have to make sure SPF can do this successfully.
1. Login to the SPF Server as the SPF service account (e.g Fabrikam\!sc)
2. Click on start and select Virtual Machine Manager Command Shell under Microsoft System Center 2012
3. Type get-vmmserver <VMM Server> e.g. get-vmmserver vmm01
Verify that get-vmmserver returns data similar to the picture above.
4. Type Get-VM | ft Name and verify that VMs running on the VMM Server is returned in the output.
5. Type Get-SCCloud | ft Name and verify that Clouds on the VMM is returned in the output
6. Close PowerShell console
If this is not returning data you should try reinstalling the VMM Console on the SPF server.
Verify that the SPF Web Service is running under the right user credentials
The way SPF executes commands against VMM will be in the context of the user under which the web service is running. A common mistake that people have is that the SPF Web Service is not running under the right account but instead it will be running as the Network Service account, which has no access in VMM.
To verify that the SPF Web Service is running under the right service account do the following:
1. Login to the SPF server as an administrator
2. Start IIS Manager
3. Expand SPF Server > Sites and verify that SPF shows in the list.
4. Select Applications Pools under connection menu
5. Verify that both the VMM and Provider Application Pools are running under the account (Identity) that was also a member of the VMM Administrators. (e.g fabrikam\!sc) as verified earlier.
If this is not the case and VMM Application Pool is running under eg. Network Service it needs to be changed to an account that has administrator role access to VMM
1. Select VMM Application Pool and Select Advanced Settings from the Action Menu
2. In the Advanced Settings select Identity and click on the … bottom to specify a user
3. Select Custom Account and click Select
4. Specify User name, Domain and Password and click OK
5. Click OK to Advance Settings.
The Web service should now be running under the right credentials to access data in VMM.
Verify that the right user for accessing SPF Web services is configured
SPF is working in such a way that a user can query the SPF Service if it’s member of local defined User Groups. When SPF is installed, four local groups are created: SPF_Admins, SPF_Providers, SPF_VMM and SPF_Usage.
In order to make WAP connect to SPF a local user should to be created on the SPF Server that is a member of all four SPF Groups.
The reason it’s recommended to use a local user and not a domain user is that WAP and SPF servers might not be in the same domain, for this reason SPF uses basic authentication to authenticate the user that accesses the web service.
To verify this do the following:
1. Logon to the SPF Server as Administrator
2. Start Computer Management
3. Select Local User and Groups
4. Locate the user you want to use for SPF or create a new user by right click Users > new user (e.g. SC)
5. Click on the user and select the “Member Of” tab.
6. Make the user member of all Groups starting with SPF_, if not add the remaning groups.
7. Click OK
Verify that WAP is registered for SPF using the right user
1. Open the WAP Admin Portal (e.g https://wap01.fabrikam.com:30091) as an Administrator
2, Select VM Clouds
3. Select the Cloud icon with the lightning
4. Click on the link under “Register System Center Service Provider Foundation”
5. Specify the SPF Web Service location, provide the local user name created earlier, which was added to the SPF local user Groups and provide the password for the user.
6. Next click on the word CLOUDS and verify that you see a VMM Server
If there is no VMM Server click on register your VMM server by clicking on USE AN EXISTING VIRTUAL..
Provide the FQDN on your VMM Server and Click OK.
Now verify that you see a VMM Server under CLOUDS.
How to verify SPF is working correctly end to end
To understand if SPF is working as expected we are going to pretend we are the Windows Azure Portal and we will query for data in SPF. By asking for specific data in SPF via the REST API we can ensure that we can extract the needed data. In order to do this we want to do the following:
- Find the user that WAP is using to communicate with SPF
- Query SPF with the SPF user.
- Query different types of data from SPF.
Pre-requisites before starting
- One or more Clouds defined in VMM
- One or more Hyper-v hosts managed by VMM.
- One or more VMs running in VMM
Find the user that WAP is using to communicate with SPF
1. Open the WAP Admin Portal (e.g https://wap01.fabrikam.com:30091)
2, Select VM Clouds
3. Select the Cloud icon with the lightning.
4. Click on the link under Register System Center Service Provider Foundation.
5. Write Down the Service URL and Username.
Query SPF with the SPF user
Now let’s try to see if we can actually talk to SPF and get SPF to return data that WAP would normally ask for.
6. Start a browser on the WAP server
7. Type in the Service URL from VM Clouds configuration and add the following /SC2012R2/VMM/Microsoft.Management.Odata.svc
8. Click continue to this website if you are using a self singed certificate.
9. Provide the user from username under VM Cloud Configuration and click Ok
You should hopefully see data on the screen similar to this:
Query different types of data from SPF
To query for data from a sub-category do the following:
1. Scroll Down and find an areas you would like to see data for. In this example we are going to use “VirtualMachines”
2. Now add the VirtualMachines (Case sensitive) to the end of the url
If you see a picture like this, you want to disable reading view in Internet Explorer.
Open Internet Options, select Content Tab and click on settings under “Feeds and Web Slices”
Uncheck “Turn on feed reading view”
Click OK and refresh the page.
3. You should now see a XML output similar to this.
4. You want to look for entry > content type > ComputerName
Note: You can also see other information about this VM by looking at the other attributes in the XML.
5. To list another Example try to add Clouds after Microsoft.Management.Odata.svc
6. In my case you can see that I have three Clouds.
7. Open VMM Console as VMM Administrator, select VMs and Services and expand Clouds
You should see the same list of Clouds as shown in SPF:
8. Open the WAP Admin Portal (e.g https://wap01.fabrikam.com:30091)
9. Select VM Clouds > Clouds and Expand the VMM Instance to see listed Clouds:
If you can’t see the same data in WAP Admin portal as in SPF there is most likely a problem connecting to SPF or SPF can’t connect to VMM.
Going over the first section making sure no steps were missing hopefully will solve the problem.
Let me know if this was helpful in any way or if any important steps are missing
If you are looking for other areas of troubleshooting WAP, have a look here: Troubleshooting Installation & Configuration of Windows Azure Pack – An Introduction
Happy Integrating WAP, SPF and VMM.