UR11 for SCOM 2012 R2 – Step by Step

 

image

 

KB Article for OpsMgr:  https://support.microsoft.com/en-us/kb/3183990

Download catalog site:  https://catalog.update.microsoft.com/v7/site/Search.aspx?q=3183990

 

 

NOTE:   I get this question every time we release an update rollup:   ALL SCOM Update Rollups are CUMULATIVE.   This means you do not need to apply them in order, you can always just apply the latest update.  If you have deployed SCOM 2012R2 and never applied an update rollup – you can go straight to the latest one available.  If you applied an older one (such as UR3) you can always go straight to the latest one!

 

Key Fixes:

  • Network discovery fails because of monitoring host crash if no paging file is set on the operating system
    When no paging file is set on the operating system, the page file size is implicitly set to 0. This causes the monitoring host to crash. This update fixes such an exception.

  • Backport PuTTY 0.64 and 0.66 updates from 2016 release
    Operations Managers ssh-based administration of UNIX/Linux computers (agent discovery and installation, upgrade, uninstallation) now supports UNIX and Linux computers that are configured to require SHA2 HMACs and those with Key Exchange Algorithm changes, as specified in RFC 4419 (Ubuntu 15.10, 16.04 LTS).

  • Update Register-SCAdvisor cmdlet for WEU workspaces
    This update adds support to register the Operations Manager Management group to workspaces in regions other than Eastern US by using the Register-SCAdvisor cmdlet. The cmdlet takes an additional optional parameter (SettingServiceUrl), which is the URL for setting the service in the region of the workspace. If it is not specified, the workspace is assumed to be in the Eastern US.

  • ACS eventschema.xml has incorrect parameter mappings for multiple audit events
    The report named Usage_-_Sensitive_Security_Groups_Changes used to say n/a\n/a for some events in the Changed By column. And in some events, the Member User column contained the account name of the user who made the change instead of the account that was added or removed. This fix resolves this issue, as the Changed By column now contains the account name of the user who made the change, and the Member User column contains the name of the added or removed account, where applicable.

  • Memory leak when monitoring network devices by using SNMPv3
    This update fixes a memory leak in Network Monitoring area that occurs while monitoring network devices by using SNMPv3.

  • Web Console user can view datawarehouse performance or SLA widget data outside of their scoped dashboard views
    This update implements verification of the logged-in user to confirm that the user has access to the opened dashboard before loading the same.

  • Downtime duration doesn't take business hour into consideration
    Business hours are being calculated even when the Business hours check box is cleared. This update resolves this issue.
    The updated RDL files are located in the following location:

    %SystemDrive%\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Reporting

    To update the RDL file, follow these steps:

    1. Go to https://MachineName/Reports_INSTANCE1/Pages/Folder.aspxMachineName //Reporting Server.
    2. On this page, go to the folder to which you want to add the RDL file. In this case, click Microsoft.SystemCenter.DataWarehouse.Report.Library.
    3. Upload the new RDL files by clicking the upload button at the top. For more information, see the Upload a File or Report (Report Manager) topic on the Microsoft Developer Network website.
  • Cisco 3172 PQ network device certification issues
    This update fixes issues in monitoring the Cisco 3172 PQ network device and its components.

  • Adding SCOM assessment links in the Operations Management Suite view in the SCOM console
    Links for Operations Manager Assessment and Pre-Configuration steps for Operations Manager Assessment are added in the Operations Management Suite Connection node under the administration pane. SCOM assessment solution in OMS is currently in private preview, please reach out to your "TAM or other Microsoft contact" to obtain access to the solution.

  • SQL Server Seed relationship with a server that is running Windows
    The dynamic members of the group are not populated if the dynamic rule is based on a hosted relationship with Windows Server. This fix enables retrieval of hosting entity.

  • Alert subscriptions are not always fired for 3 state monitors
    This update allows for configuration of a 3 state monitor to raise alerts; whose severity is in sync with monitor’s health state. One can create a subscription to be notified on modification of alert severity. Even if the monitor’s state keeps toggling between warning and critical, severity of the alert keeps being updated, and a notification fired on an update to the severity of an alert.

  • When you connect SCOM to OMS, Availability monitors health state of some management servers changes to Warning state
    If OMS workspace is configured to collect certain event logs and if those event logs are not present on the management servers that are connected to that workspace, the “Availability” monitor’s health state on those management servers that are used to change the Warning state. This state change scenario is fixed. This prevents the switch to Warning state.

  • RunAs Account password expiration does not work with Active directory Password Settings Objects that breaks the validation of LOCAL User Accounts
    Local Accounts could not be validated by using ADSystemInfo. Therefore, when any local account is added into RunAs account, an Error event is logged in Event Viewer for an exception in validating the local account. After this update, local accounts are validated.

  • MPB Entries in Catalog database for the VersionIndependentGuid column is updated
    This update enables the SCOM console to show the correct mpb installation status in the management server when it tries to connect to an online catalog to update mpbs.

  • If the first try of importing MPB failed then re-importing the MPB was not possible until the SCOM console is closed and reopened
    This update enables re-download and installation of an mpb, without closing and reopening the SCOM console, even if the first try installation of that mpb failed because of a dependency issue.

  • Change of the displayname field for a group in a sealed or unsealed management pack
    Renaming a group through PowerShell cmdlets was not displaying the new group name in the SCOM console. This update resolves this issue and the renaming of a group correctly displays the renamed group name in the SCOM console.

New Linux operating system versions supported
  • Ubuntu Linux 16.04 LTS (x86 and x64) is now supported in System Center 2012 R2 Operations Manager.
Issues that are fixed in the UNIX and Linux management packs
  • During UNIX/Linux computer discovery, the GetOSVersion.sh script is run with sudo elevation if a sudo-enabled user is selected for Discovery. This update prevents the GetOSVersion.sh script from being run with sudo elevation and does not have to be authorized in the /etc/sudoers folder.

  • Scripts executed by the ExecuteScript method in Management Packs always run from the /tmp folder. With this update, the temporary folder for scripts is now configurable. To use another folder, update the symbolic link to link to a temporary folder of your choice:

    /etc/opt/microsoft/scx/conf/tmpdir

  • UNIX or Linux computers together with sshd versions that implement the Key Exchange Algorithms described in RFC 4419, such as Ubuntu 15.10, cannot be discovered with the Discovery Wizard.

  • Network statistics collected on AIX servers are reset when another tool such as NetStat is also used.

  • Physical disks are shown incorrectly as offline if an LVM snapshot is taken.

 

 

 

Lets get started.

From reading the KB article – the order of operations is:

  1. Install the update rollup package on the following server infrastructure:
    • Management servers
    • Audit Collection servers 
    • Gateway servers
    • Web console server role computers
    • Operations console role computers
  2. Apply SQL scripts.
  3. Manually import the management packs.
  4. Update Agents

Additionally, we will add the steps to update Linux management packs and agents.

 

1. Management Servers

image

Since there is no RMS anymore, it doesn’t matter which management server I start with.  There is no need to begin with whomever holds the “RMSe” role.  I simply make sure I only patch one management server at a time to allow for agent failover without overloading any single management server.

I can apply this update manually via the MSP files, or I can use Windows Update.  I have 3 management servers, so I will demonstrate both.  I will do the first management server manually.  This management server holds 3 roles, and each must be patched:  Management Server, Web Console, and Console.

The first thing I do when I download the updates from the catalog, is copy the cab files for my language to a single location:

 

image

 

Then extract the contents:

 

image

 

Once I have the MSP files, I am ready to start applying the update to each server by role.

 

***Note: You MUST log on to each server role as a Local Administrator, SCOM Admin, AND your account must also have System Administrator role to the SQL database instances that host your OpsMgr databases.

 

My first server is a management server, and the web console, and has the OpsMgr console installed, so I copy those update files locally, and execute them per the KB, from an elevated command prompt:

 

image

 

This launches a quick UI which applies the update.  It will bounce the SCOM services as well.  The update usually does not provide any feedback that it had success or failure. 

 

You can check the application log for the MsiInstaller events to show completion:

 

Log Name: Application
Source: MsiInstaller
Date: 8/31/2016 9:01:13 AM
Event ID: 1036
Description:
Windows Installer installed an update. Product Name: System Center Operations Manager 2012 Server. Product Version: 7.1.10226.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Update Name: System Center 2012 R2 Operations Manager UR11 Update Patch. Installation success or error status: 0.

 

You can also spot check a couple DLL files for the file version attribute. 

 

image

 

Next up – run the Web Console update:

 

image

 

This runs much faster.   A quick file spot check:

 

image

 

 

Lastly – install the console update (make sure your console is closed):

 

image

 

A quick file spot check:

 

image

 

 

Additional Management Servers:

image

 

I now move on to my additional management servers, applying the server update, then the console update and web console update where applicable.

On this next management server, I will use the example of Windows Update as opposed to manually installing the MSP files.  I check online, and make sure that I have configured Windows Update to give me updates for additional products: 

 

image

 

The applicable updates show up under optional – so I tick the boxes and apply these updates.

 

image

 

After a reboot – go back and verify the update was a success by spot checking some file versions like we did above.

 

 

Updating ACS (Audit Collection Services)

You would only need to update ACS if you had installed this optional role.

On any Audit Collection Collector servers, you should run the update included:

 

image

 

image

 

A spot check of the files:

 

image

 

 

 

 

Updating Gateways:

image

 

I can use Windows Update or manual installation.

image

 

The update launches a UI and quickly finishes.

I was prompted for a reboot.

image

 

Then I will spot check the DLL’s:

 

image

 

I can also spot-check the \AgentManagement folder, and make sure my agent update files are dropped here correctly:

 

image

 

***NOTE: You can delete any older UR update files from the \AgentManagement directories. The UR’s do not clean these up and they provide no purpose for being present any longer.

 

 

 

2. Apply the SQL Scripts

 

In the path on your management servers, where you installed/extracted the update, there are two SQL script files: 

%SystemDrive%\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\SQL Script for Update Rollups

(note – your path may vary slightly depending on if you have an upgraded environment or clean install)

 

image

First – let’s run the script to update the OperationsManagerDW (Data Warehouse) database.  Open a SQL management studio query window, connect it to your Operations Manager DataWarehouse database, and then open the script file (UR_Datawarehouse.sql).  Make sure it is pointing to your OperationsManagerDW database, then execute the script.

You should run this script with each UR, even if you ran this on a previous UR.  The script body can change so as a best practice always re-run this.

If you see a warning about line endings, choose Yes to continue.

image

 

Click the “Execute” button in SQL mgmt. studio.  The execution could take a considerable amount of time and you might see a spike in processor utilization on your SQL database server during this operation.

You will see the following (or similar) output:   “Command(s) completes successfully”

 

image

Next – let’s run the script to update the OperationsManager (Operations) database.  Open a SQL management studio query window, connect it to your Operations Manager database, and then open the script file (update_rollup_mom_db.sql).  Make sure it is pointing to your OperationsManager database, then execute the script.

You should run this script with each UR, even if you ran this on a previous UR.  The script body can change so as a best practice always re-run this.

 

image

 

Click the “Execute” button in SQL mgmt. studio.  The execution could take a considerable amount of time and you might see a spike in processor utilization on your SQL database server during this operation.  

I have had customers state this takes from a few minutes to as long as an hour. In MOST cases – you will need to shut down the SDK, Config, and Monitoring Agent (healthservice) on ALL your management servers in order for this to be able to run with success.

 

You will see the following (or similar) output: 

image

or

image

 

 

IF YOU GET AN ERROR – STOP!   Do not continue.  Try re-running the script several times until it completes without errors.  In a production environment with lots of activity, you will almost certainly have to shut down the services (sdk, config, and healthservice) on your management servers, to break their connection to the databases, to get a successful run.

Technical tidbit: Even if you previously ran this script in any previous UR deployment, you should run this again in this update, as the script body can change with updated UR’s.

 

3. Manually import the management packs

image

 

There are 55 management packs in this update!   Most of these we don’t need – so read carefully.

The path for these is on your management server, after you have installed the “Server” update:

\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Management Packs for Update Rollups

However, the majority of them are Advisor/OMS, and language specific.  Only import the ones you need, and that are correct for your language.  I will remove all the MP’s for other languages (keeping only ENU), and I am left with the following:

image

 

What NOT to import:

The Advisor MP’s are only needed if you are using Microsoft Operations Management Suite cloud service, (Previously known as Advisor, and Operations Insights).

The APM MP’s are only needed if you are using the APM feature in SCOM.

Note the APM MP with a red X. This MP requires the IIS MP’s for Windows Server 2016 which are in Technical Preview at the time of this writing. Only import this if you are using APM *and* you need to monitor Windows Server 2016. If so, you will need to download and install the technical preview editions of that MP from https://www.microsoft.com/en-us/download/details.aspx?id=48256

The TFS MP bundle is only used for specific scenarios, such as DevOps scenarios where you have integrated APM with TFS, etc. If you are not currently using these MP’s, there is no need to import or update them. I’d skip this MP import unless you already have these MP’s present in your environment.

However, the Image and Visualization libraries deal with Dashboard updates, and these always need to be updated.

I import all of these shown without issue.

 

 

 

4. Update Agents

image

Agents should be placed into pending actions by this update for any agent that was not manually installed (remotely manageable = yes):  

One the Management servers where I used Windows Update to patch them, their agents did not show up in this list.  Only agents where I manually patched their management server showed up in this list.  FYI.   The experience is NOT the same when using Windows Update vs manual.  If yours don’t show up – you can try running the update for that management server again – manually.

 

image

If your agents are not placed into pending management – this is generally caused by not running the update from an elevated command prompt, or having manually installed agents which will not be placed into pending.

In this case – my agents that were reporting to a management server that was updated using Windows Update – did NOT place agents into pending.  Only the agents reporting to the management server for which I manually executed the patch worked.

I manually re-ran the server MSP file manually on these management servers, from an elevated command prompt, and they all showed up.

You can approve these – which will result in a success message once complete:

 

image

 

Soon you should start to see PatchList getting filled in from the Agents By Version view under Operations Manager monitoring folder in the console:

 

image

 

 

 

5. Update Unix/Linux MPs and Agents

 

image

 

The current Linux MP’s can be downloaded from:

https://www.microsoft.com/en-us/download/details.aspx?id=29696

 

7.5.1060.0 is current at this time for SCOM 2012 R2 UR11.

****Note – take GREAT care when downloading – that you select the correct download for SCOM 2012 R2. You must scroll down in the list and select the MSI for 2012 R2:

 

image

 

Download the MSI and run it.  It will extract the MP’s to C:\Program Files (x86)\System Center Management Packs\System Center 2012 R2 Management Packs for Unix and Linux\

Update any MP’s you are already using.   These are mine for RHEL, SUSE, and the Universal Linux libraries. 

 

image

 

NOTE:   Upon first import – you might see that “Linux Operating System Library” (Microsoft.Linux.Library.mp) file fails to import.  If this happens, simply make sure you have imported version 7.5.1060.0 of UNIX/Linux Core Library (Microsoft.Unix.Library.mp) FIRST, then you can import “Linux Operating System Library” (Microsoft.Linux.Library.mp) without issue.

 

You will likely observe VERY high CPU utilization of your management servers and database server during and immediately following these MP imports.  Give it plenty of time to complete the process of the import and MPB deployments.

Next – you need to restart the “Microsoft Monitoring Agent” service on any management servers which manage Linux systems.  I don’t know why – but my MP’s never drop/update in the \Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\AgentManagement\UnixAgents\DownloadedKits folder until this service is restarted.

Next up – you would upgrade your agents on the Unix/Linux monitored agents.  You can now do this straight from the console:

 

image

 

image

 

You can input credentials or use existing RunAs accounts if those have enough rights to perform this action.

Finally:

image

 

 

 

6. Update the remaining deployed consoles

 

image

This is an important step.  I have consoles deployed around my infrastructure – on my Orchestrator server, SCVMM server, on my personal workstation, on all the other SCOM admins on my team, on a Terminal Server we use as a tools machine, etc.  These should all get the matching update version.

 

 

 

Review:

image

Now at this point, we would check the OpsMgr event logs on our management servers, check for any new or strange alerts coming in, and ensure that there are no issues after the update.

Known issues:

See the existing list of known issues documented in the KB article.

1.  Many people are reporting that the SQL script is failing to complete when executed.  You should attempt to run this multiple times until it completes without error.  You might need to stop the Exchange correlation engine, stop all the SCOM services on the management servers, and/or bounce the SQL server services in order to get a successful completion in a busy management group.  The errors reported appear as below:

——————————————————
(1 row(s) affected)
(1 row(s) affected)
Msg 1205, Level 13, State 56, Line 1
Transaction (Process ID 152) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Msg 3727, Level 16, State 0, Line 1
Could not drop constraint. See previous errors.
——————————————————–