Lessons learned - The OpsMgr 2007 R2 Universal Connector

Last week I was asked to help a customer with the configuration of an OpsMgr 2007 R2 connector. Because I had not much hands on experience with the installation and configuration of the OpsMgr 2007 R2 Connectors I installed an configured the Universal Connector in my own test/demo environment.

And because I learned some new things during this exercise I thought it would be a good idea to share some of my experiences. Warning, this turned out to be become a long blogpost. Stop reading here if you can only read short (twitter) messages Winking smile

Environment:

  • OpsMgr 2007 R2 CU3
  • OpsMgr 2007 R2 RMS server running on Windows 2008 R2
  • All-In-One RMS with SQL 2008 R2
  • SUSE Linux Enterprise Server (Linux suse10 2.6.16.46-0.12-default)

 

Downloads:

I started downloading the free connectors from Microsoft Downloads. This release of the Operations Manager 2007 R2 Connectors includes the following Connectors:

  • Microsoft System Center Operations Manager 2007 R2 Connector for IBM Tivoli Enterprise Console
  • Microsoft System Center Operations Manager 2007 R2 Connector for HP OpenView Operations for Unix
  • Microsoft System Center Operations Manager 2007 R2 Connector for HP OpenView Operations for Windows
  • Microsoft System Center Operations Manager 2007 R2 Connector for BMC Remedy ARS
  • Microsoft System Center Operations Manager 2007 R2 Universal Connector

 

Documentation:

If you extract the downloaded SCInterop_R2_RTM.zip file there is a Docs folder with 4 files of documentation you should be READING before installing the Connectors.

  • OM2007_Connectors.doc
  • OM2007_ConnectorsRn.htm
  • OM2007_MP_Connector.doc
  • SCOM2007R2_UniversalConnector_DevelopingAnIntegration.doc

 

Tools:

I used the following tools to install and configure the Universal Provider on my SLES10 system:

  • Putty (PuTTY is an SSH and telnet client)
  • WinSCP (WinSCP is an open source free SFTP client, SCP client, FTPS client and FTP client for Windows)
    I used this tool to copy the scx-<build-number>.sles.10.x86.rpm (Interop Core package) installation files from my Windows machine to the SLES10 machine.

 

You should start reading the Release Notes (OM2007_ConnectorsRn.htm)  thoroughly before you install or upgrade any Operations Manager 2007 R2 Connectors. The next step is reading the Operations Manager 2007 R2 Connectors Deployment Guide (OM2007_Connectors.doc). Here you can check the System Requirements.  In my test/demo environment I’ve installed the Connector Service and Configuration UI on a Windows Server 2008 R2 system without any issues. But be aware this is not supported (yet).

Other documentation I used to get ready for the installation is this blogpost from our famous Kevin Holman Installing the OpsMgr R2 Universal Connector.

Operations Manager 2007 Connectors Architecture:

UniversalConnector

 

High-Level Installation Steps:

  1. Check Pre-requisites on both systems.
  2. Install Universal Provider on SUSE10 server (always first start with the Provider)
  3. Install Universal Connector Service and Configuration on Windows Server 2008 R2 Server
  4. Configure the Universal Connector

 

For the detailed installation steps see the Operations Manager 2007 R2 Connectors Deployment Guide but I’ll show some tips and tricks I used to get the Universal Connector installed and configured.

 

Tips & Tricks:

Step 1. Check Pre-requisites on both systems.

How do you check if WSMan is installed on a Windows system?

You can check if WSMan is installed using the next command from the command prompt:

winrm identify

image

So on my Windows 2008 R2 RMS (where I’ll be installing the Universal Connector Service and UI) WSMan is running.

How can check if WSMan is running on the remote SLES10 machine?

Keep in mind that during the install of the SCX agent (Interop core) first  the wsman components (Open Pegasus) will be installed and later the Univeral provider will be installed on top of the SCX agent.

So it would be “normal” not having wsman installed on the remote SLES10 machine. But in case you want to check anyway. You could check for the cimserver process on the SLES10 machine using the next command in Putty:

ps –eaf | grep cimserver

image

The winrm also has a remote parameter. winrm id –remote:nameofremoteserver.com

Let’s try to run this from the RMS server.

image

It seems we don’t have the correct permissions to do this. Let’s try to use the winrm’s INVOKE verb. More info on the use of the INVOKE verb can be found at Daniele Muscetta’s blogpost about winrm and invoke.

Run the following command from a Windows machine:

winrm invoke ExecuteCommand https://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem?__cimnamespace=root/scx @{command="ps -eaf";timeout="60"} -username:root -password:Password -auth:basic -r:https://suse10:1270/wsman -skipCACheck -encoding:UTF-8

image

There we have it, all running processes on the SLES10 machine using WSMan. Keep in mind this test was run AFTER I had installed the Interop core which also installs Open Pegasus.

 

Do I need to install the 32-bit version of the Microsoft Visual C++ 2008 Redistributable Package (x86) on a 64-bit Windows 2008 R2 server?

Yes you need to install the x86 version on a 64-bit OS.

 

Step 2. Install Universal Provider on SUSE10 server (always first start with the Provider)

How do I copy the Universal Provider files to my SLES10 server?

Use WinSCP to connect from your Windows server, where you have the installation files for the connectors to the SLES10 server. Copy the correct OS version files (in my case I copied the files from the Linux_SUSE_10.0_x86_32 folder) to a location (/tmp) on the remote SLES10 server.

clip_image002

 

How do I install the Universal Provider on my SLES10 server?

Use Putty to make a connection to the remote SLES10 machine. Logon with an account which has enough privileges to install the software (I used the root account for the installation of the Universal Provider).

The first part of the installation is the installation of the the SCX agent first (Interop core) the wsman components (Open Pegasus)

run the following command in your Putty session: rpm –i /tmp/Linux_SUSE_10.0_x86_32/scx-1.0.4-248.sles.10.x86.rpm

image

The second part of the installation is the installation of the Universal Provider.

Run the next command in your Putty session: Rpm -i MSFTscinteropUnv-6.17000-58.suse.10.x86.rpm

image

You can check the result of the installation in the /tmp/scinterop_install.out file using cat or WinSCP if you want.

 

Step 3. Install Universal Connector Service and Configuration on Windows Server 2008 R2 Server

Install the Universal Connector and UI using an elevated command prompt.

Which Features do I need to select during the installation of the Universal Connector and UI?

Select Entire feature will be installed on local harddrive.

image

Remark: You can change the location where the Universal Connector Service is installed if you want.

 

Which account be used for the Connector Service?

  • On the Connector Service Login page, type in the credentials for the connector service. The connector service will be configured to run under these credentials. This will be used to connect to the SDK on the RMS, and will be used to authenticate to the provider via WSMAN. I used my SDK service account for this, since it already has access to the OpsMgr SDK. Added this account to the local admins group on Server running the Connector Service during the prerequisites stage.
  • Remark: OM_SDK account needs to be member of the Operations Manager Administrators Role.

 

After the installation of the Universal Connector Service a new database SCInterop is created.

One step in the configuration is granting the Connector Service Account (SDK account) enough permissions for the SCInterop database. The SDK Config account need db_owner permissions.

image

Now it’s time to configure the Universal Connector.

image

image

Which User name do I use for the WS-Man server credentials?

Enter the next settings:

  • Operations Manager Server (RMS name to connect to the SDK)
  • Universal Provider (Provider server NetBIOS name – NOT FQDN)
  • WS-Man Server credentials (this is an account that has rights to the provider server via WSMan)

 

image

Instead of using the root account on the SLES10 server where I installed the Universal Provider I created a scxuser account.

The scxuser is member of the following groups on the SLES10 server:

image

The next step in the Configuration is testing the connection.

image

The first time I tested the connection I got an error message.

image

Before clicking on Yes to continue the configuration I opened a command prompt to test the connection using the scxuser account.

Run the following command from the command prompt:
winrm enumerate https://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem?__cimnamespace=root/scx -username:scxuser -password:Password -r:https://suse10:1270/wsman -auth:basic -skipCACheck -encoding:UTF-8

image

There seems something wrong with the SSL certificate.

The Connector requires the use of certificates to validate the authenticity of the server on which the Interop Provider is running. The Connector does not work until the certificate has been transferred and correctly imported to the server on which Connector is running (Windows 2008 R2 RMS Server) from the server on which the Interop Provider (SLES10 Server) is running. During the Interop Provider installation, a self-signed certificate is generated and stored in the Interop Provider installation directory. The Connector Certificate Retrieval and Installation wizard retrieves the certificate and automatically installs it on the server on which the Connector is running. Installing the Connector certificate at installation is optional. However, the Connector certificate must be installed on the server on which the Connector is running before the Connector service is started. If you do not install the Interop Provider certificate, the Connector cannot communicate with the server on which the Interop Provider is running.

Fixed it by following the next steps:

1. Copied the scx-host-suse10.pem file to the RMS server

clip_image001

2. Runned scxcertconfig -sign scx-host-suse10.pem scx_new.pem on RMS

clip_image002

3. Copied the scx_new.pem file to the SUSE10 box (Provider) and renamed it to scx-host-suse10.pem

clip_image003

4. Restarted scxadmin –restart (on the SLES10 server)

image

5. Tested WSMan connection to provider from RMS.

In the Universal Connector folder there is a tool called scicert.exe which can be used to test the connection.

image

Run the following command from the command prompt:

scicert suse10 scxuser Password "OpsMgr Universal Connector" test

image

Seems to be working ok, right now Smile

You can also use the winrm enumerate command if you want to:

winrm enumerate https://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem?__cimnamespace=root/scx -username:scxuser -password:Password -r:https://suse10:1270/wsman -auth:basic -skipCACheck -encoding:UTF-8

image

 

We now have the Universal Connector installed and configured, now we need to test if OpsMgr Alerts will be forwarded from OpsMgr to the Universal Provider on the SLES10 server.

 

High-Level Steps to test the Universal Connector.

  1. Add to add a new subscriptions for alerts that you want to send to the connector.
  2. Create a test event rule
  3. Manually emulate an EMS (Enterprise Management System)

 

Step 1. Add  a new subscription for alerts that you want to send to the connector.

Go to Administration Pane and select Internal Connectors folder. Select the OpsMgr Universal Connector.

image

Double click on the OpsMgr Universal Connector. In the properties of the OpsMgr Universal Connector enter a Subscription Name and optionally a Description.

image

Click Next and configure the Groups criteria

image

Click Next and configure the Targets.

image

Click Next and configure the Alert Criteria

image

And in the last screen click OK to save your Subscription.

image

 

Step 2. Create a Test Event Rule.

Just follow the normal steps to create an Event Alert Rule which get’s triggered by eventid 999.

image

image

Every time an event 999 is created an Alert is generated.

Just use the EventCreate.exe tool to create an Event 999 from the command prompt:

eventcreate.exe /T ERROR /ID 999 /L APPLICATION /D "Testing Universal Connector"

Result:

image

 

Before we look at what happens on the Remote Universal Provider we need to have some background info on the New Alert Data workflow.

 

New Alert Data Flow in the Universal Connector:

  1. The Operations Manager alert is forwarded to remote system in either XML or Key=Value Pair(evt) file format (I configured to use XML)
  2. The universal provider on the remote system creates a file (xml or property file) in a specified directory
  3. The custom integration logic picks up file from specified directory and inserts data into the remote system application
  4. The custom integration logic creates a file in a specified directory with specific data to update the Operations Manager alert, i.e. TicketID
  5. The universal provider picks up custom integration file and sends it back to Operations Manager to update the alert
  6. The Operations Manager alert is updated

 

Step 1 and 2. The Operations Manager alert is forwarded to remote system in either XML or Key=Value Pair(evt) file format and The universal provider on the remote system creates a file (xml or property file) in a specified directory

We can check if the OpsMgr Alert is forwarded to the remote Universal Provider, which I configured to be in XML format, by opening WinSCP and look in the following folder: /opt/microsoft/scx/UnvEvents/FromOpsMgr

image

Let’s create a new Alert by creating a new 999 EventID and check if there will created an XML file in the /opt/microsoft/scx/UnvEvents/FromOpsMgr folder

image

image

Now look at the remote folder and check if we see the xml file on the Universal Provider.

image

Let’s check the contents of the xml file.

 

<?xml version="1.0" encoding="utf-8"?> <UNVEvent> <AlertId>c16964d1-b01d-4e44-99c9-d892d28fe680</AlertId> <ComputerDomain>DEMO</ComputerDomain> <ComputerName>OPSMGRR2RMS</ComputerName> <Description>Event Description: Testing Universal Connector</Description> <EventType>0</EventType> <ManagementGroupName>DEMOMG</ManagementGroupName> <ManagementPack>Stefan.Test.Universal.Connector</ManagementPack> <ManagementServer>opsmgrr2rms</ManagementServer> <ModifiedBy>DEMO\OM_Admin</ModifiedBy> <MonitoringClassName>Microsoft.Windows.Computer</MonitoringClassName> <MonitoringObjectHealthState>Success</MonitoringObjectHealthState> <MonitoringObjectInMaintenanceMode>False</MonitoringObjectInMaintenanceMode> <MonitoringObjectName>OPSMGRR2RMS.DEMO.STRANGER</MonitoringObjectName> <Name>Stefan - Test Universal Connector Event Rule</Name> <Priority>Normal</Priority> <ProblemId>afcc789a-d227-37a2-79ff-70c6cf469eba</ProblemId> <RepeatCount>0</RepeatCount> <ResolutionState>255</ResolutionState> <RuleName>MomUIGeneratedRule75cfefecc3264771af35f117e44a64ff</RuleName> <RuleTarget>Microsoft.Windows.Computer</RuleTarget> <Severity>2</Severity> <TimeOfLastEvent>2011-08-15T16:48:43.557Z</TimeOfLastEvent> <WebConsoleUrl>https://OPSMGRR2RMS:51908/default.aspx?DisplayMode=Pivot&amp;AlertID=c16964d1-b01d-4e44-99c9-d892d28fe680</WebConsoleUrl> </UNVEvent>

Let’s also check if something has changed on the OpsMgr side.

image

We don’t see a new Ticked ID or Owner change. The only indication we see is that the History has some info.

image

 

Step 3. The custom integration logic picks up file from specified directory and inserts data into the remote system application.

Now the Universal EMS should pick up the xml file from the /opt/microsoft/scx/UnvEvents/FromOpsMgr and change this data in the EMS (Enterprise Management System). We can do the same manually if we want to test this.

Steps:

1. Create a new directory called "<Provider Install Directory>\UnvEvents\<MgmtGrpName>\ using WinSCP on the SLES10 server.

clip_image001[4]

clip_image002[4]

clip_image003[4]

2. Create a directory called "<Provider Install Directory>\UnvEvents\<MgmtGrpName>\EMS". This directory will contain a evt/xml file with all the necessary information about the alert that is in sync with the connector. If you close the alert in OpsMgr this file will be deleted, acknowledging that the alert was closed.

clip_image004[4]

3. You can manipulate the evt/xml file in the EMS directory then drop it into the "<Provider Install Directory>\UnvEvents\<MgmtGrpName>" directory to send updates back to the OpsMgr alert.

To send an update back to OpsMgr the EventType variable must always be set to 1 (e.g.; EventType=1 or <EventType>1</EventType>).

Fields that can be update on the OpsMgr alert from the EMS side:

  • EventType
  • OwnerName
  • CustomField1 thru CustomField10
  • ResolutionState

Let's update the Alert:

Copy file c16964d1-b01d-4e44-99c9-d892d28fe680.1.xml from /opt/microsoft/scx/UnvEvents/FromOpsMgr to /opt/microsoft/scx/UnvEvents/DEMOMG/EMS folder

 

cp /opt/microsoft/scx/UnvEvents/FromOpsMgr/c16964d1-b01d-4e44-99c9-d892d28fe680.1.xml /opt/microsoft/scx/UnvEvents/DEMOMG/EMS

image

 

Edit c16964d1-b01d-4e44-99c9-d892d28fe680.1.xml

<?xml version="1.0" encoding="utf-8"?> <UNVEvent> <AlertId>c16964d1-b01d-4e44-99c9-d892d28fe680</AlertId> <ComputerDomain>DEMO</ComputerDomain> <ComputerName>OPSMGRR2RMS</ComputerName> <Description>Event Description: Testing Universal Connector</Description> <EventType>0</EventType> <ManagementGroupName>DEMOMG</ManagementGroupName> <ManagementPack>Stefan.Test.Universal.Connector</ManagementPack> <ManagementServer>opsmgrr2rms</ManagementServer> <ModifiedBy>DEMO\OM_Admin</ModifiedBy> <MonitoringClassName>Microsoft.Windows.Computer</MonitoringClassName> <MonitoringObjectHealthState>Success</MonitoringObjectHealthState> <MonitoringObjectInMaintenanceMode>False</MonitoringObjectInMaintenanceMode> <MonitoringObjectName>OPSMGRR2RMS.DEMO.STRANGER</MonitoringObjectName> <Name>Stefan - Test Universal Connector Event Rule</Name> <Priority>Normal</Priority> <ProblemId>afcc789a-d227-37a2-79ff-70c6cf469eba</ProblemId> <RepeatCount>0</RepeatCount> <ResolutionState>0</ResolutionState> <RuleName>MomUIGeneratedRule75cfefecc3264771af35f117e44a64ff</RuleName> <RuleTarget>Microsoft.Windows.Computer</RuleTarget> <Severity>2</Severity> <TimeOfLastEvent>2011-08-15T16:48:43.557Z</TimeOfLastEvent> <WebConsoleUrl>https://OPSMGRR2RMS:51908/default.aspx?DisplayMode=Pivot&amp;AlertID=c16964d1-b01d-4e44-99c9-d892d28fe680</WebConsoleUrl> </UNVEvent>

<?xml version="1.0" encoding="utf-8"?> <UNVEvent> <AlertId>c16964d1-b01d-4e44-99c9-d892d28fe680</AlertId> <ComputerDomain>DEMO</ComputerDomain> <ComputerName>OPSMGRR2RMS</ComputerName> <Description>Event Description: Testing Universal Connector</Description> <EventType>1</EventType> <ManagementGroupName>DEMOMG</ManagementGroupName> <ManagementPack>Stefan.Test.Universal.Connector</ManagementPack> <ManagementServer>opsmgrr2rms</ManagementServer> <ModifiedBy>DEMO\OM_Admin</ModifiedBy> <MonitoringClassName>Microsoft.Windows.Computer</MonitoringClassName> <MonitoringObjectHealthState>Success</MonitoringObjectHealthState> <MonitoringObjectInMaintenanceMode>False</MonitoringObjectInMaintenanceMode> <MonitoringObjectName>OPSMGRR2RMS.DEMO.STRANGER</MonitoringObjectName> <Name>Stefan - Test Universal Connector Event Rule</Name> <Priority>Normal</Priority> <ProblemId>afcc789a-d227-37a2-79ff-70c6cf469eba</ProblemId> <RepeatCount>0</RepeatCount> <ResolutionState>255</ResolutionState> <RuleName>MomUIGeneratedRule75cfefecc3264771af35f117e44a64ff</RuleName> <RuleTarget>Microsoft.Windows.Computer</RuleTarget> <Severity>2</Severity> <TimeOfLastEvent>2011-08-16T10:44:19.830Z</TimeOfLastEvent> <WebConsoleUrl>https://OPSMGRR2RMS:51908/default.aspx?DisplayMode=Pivot&amp;AlertID=1fbc6201-773f-4680-a120-e6e7e684cd8e</WebConsoleUrl> <OwnerName>EMS</OwnerName> <CustomField1>Manual Test of EMS</CustomField1> </UNVEvent>

Drop it into the "/opt/microsoft/scx/UnvEvents/DEMOMG

cp /opt/microsoft/scx/UnvEvents/DEMOMG/EMS/c16964d1-b01d-4e44-99c9-d892d28fe680.1.xml /opt/microsoft/scx/UnvEvents/DEMOMG

 

image

Is gone in seconds

clip_image014

imageimage

Works ok :-)

 

I also created a PowerShell script which retrieves the remote xml file so you can check what has been forwarded from the Universal Connector to the Remote Universal Provider. But I need some more time to improve this script.

Here is a first example on how I started. If I’ve more time I’ll create a blogpost on how to create a PowerShell EMS tool.

#use winrm with invoke to retrieve the last xml file from the remote Universal Provider [string]$AlertFromUnvConn =  cmd /c 'winrm invoke ExecuteShellCommand https://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem?__cimnamespace=root/scx @{command="ls -1t /opt/microsoft/scx/UnvEvents/FromOpsMgr/*.xml | head -1 | xargs cat";timeout="60"} -username:root -password:Password -auth:basic -r:https://suse10:1270/wsman -skipCACheck -encoding:UTF-8' $subject = $AlertFromUnvConn

#Parse the result so we can use this fix the returned XML if ($subject -cmatch '(?si)<UNVEvent\b[^>]*>(.*?)</UNVEvent>') {     $result = $matches[0] } else {     $result = '' }

#Convert result to genuine xml [xml]$xml = $result

#Show Result $xml.UNVEvent | format-list

Result:

image

 

Have fun!

Tweet