OpsMgr 2007 R2 CU4 rollup hotfix ships – and my experience installing it


****NOTE OpsMgr 2007 R2 CU5 is now shipped and this is an old article.

http://blogs.technet.com/b/kevinholman/archive/2011/08/03/opsmgr-2007-r2-cu5-rollup-hotfix-ships-and-my-experience-installing-it.aspx

 

 

 

image

 

The KB article describing the fixes, changes, and instructions:

http://support.microsoft.com/kb/2449679

Get it from the download Center:

http://www.microsoft.com/downloads/details.aspx?FamilyID=39a871e0-d996-4568-be5d-2edb6f539330

List of all OpsMgr R2 Cumulative Updates:

http://support.microsoft.com/kb/2453149

 

***Caution:  There is a known issue documented in the KB article where applying this CU4 on existing OpsMgr agents can cause an unexpected restart of several non-OpsMgr services.  This is caused by the Server 2008 Windows Installer RestartManager trying to suppress a reboot as we attempt to update a locked file.  This can potentially cause application outages as the services for other Microsoft core OS components and some 3rd party application services might be restarted.  This only affects agents running Windows 2008 and Windows 2008 R2.  For this reason, you might consider skipping this update and waiting for CU5.  Or, consider applying this update ONLY to your OpsMgr server roles, and rejecting any agent updates until the next CU.  Here is the actual text from the KB:

  • Restart of non-Operations Manager services
    In certain cases, non-Operations Manager services may be restarted when the Operations Manager agent is updated. This issue only affects computers that are running Windows Server 2008 or Windows Server 2008 R2. We recommend that you update agents at a time when a service restart or a server restart is acceptable. Or, only update agents that are experiencing one or more of the agent-related issues that are mentioned in the list of resolved issues. This issue will be addressed in an upcoming cumulative update.

Make sure you see the other known issues and troubleshooting section both in the KB and below at the bottom of this article.

 

 

The release notes in the KB article above cover the fixes that are included.  Of course, you should evaluate the fixes provided in ANY update and decide if you are impacted and if it makes sense for you to apply.  The most common issue I have seen is where the RMS does not recover after the SQL server hosting the OpsDB or DWDB is rebooted.  Also – ESPECIALLY if you are considering migrating to SQL 2008R2, this update will be required. 

 

 

Here are the high level fixes:

    • Feature addition: SQL 2008 R2 Database Upgrade Support.

    • Feature Addition: Health Service automatic recovers added for SQL Server Failure (Not enabled by default. See details below for how to enable)

    • PRO Tips did not function correctly after CU3 was installed

    • Alert View not working correctly when alert source contains both monitors and rules

    • Event description is not being collected by Azure MP. In order for the event description to be collected properly, the Windows Azure application needs to be build using Windows Azure SDK 1.3

    • MonitoringHost stops responding in some cases for agents running on Windows Server 2003 SP2

    • Diagram View performance is slow when a user role is scoped to many groups

    • Hyperlinks in Knowledge tab of alert properties are not active in web console

 

 

Let’s Roll:

 

So – first – I download it.  The hotfix is about 1.26GB in size.  Yes, gigabytes.

Now – before your heart rate starts rising…. understand… this update is the second CU which combines the Cross Plat CU with the OpsMgr CU.  (CU3 did this as well)  Aligning these is a very good thing – but it ends up increasing the size of the initial download.  No worries though – I will demonstrate later how to only have to copy specific files to lessen the impact of distributing this update to all your management servers and gateways, if copying a 1.26GB file around is a problem for you.

 

Next step – READ the documentation… understand all the steps required, and formulate the plan.

 

 

I build my deployment plan based on the release notes in the KB article.  My high level plan looks something like this:

  1. Backup the Operations and Warehouse databases, and all unsealed MP’s.
  2. Apply the hotfix to the RMS
  3. Run the SQL script(s) update against the OpsDB AND Warehouse DB.
  4. Import the updated management packs provided.
  5. Apply the hotfix to all secondary Management Servers.
  6. Apply the hotfix to my Gateway Servers.
  7. Apply the hotfix to my agents by approving them from pending
  8. Apply the hotfix my dedicated consoles (Terminal servers, desktop machines, etc…)
  9. Apply the hotfix to my Web Console server
  10. Apply the hotfix to my Audit collection servers
  11. Update manually installed agents…. well, manually.

 

Ok – looks like 11 easy steps.  This order is not set in stone – it is a recommendation based on logical order, from the release notes.  For instance – if you wanted to update ALL your infrastructure before touching any agent updates – that probably makes more sense and would be fine.

****Requirement – as a required practice for a major update/hotfix, you should log on to your OpsMgr role servers using a domain user account that meets the following requirements:

  • OpsMgr administrator role
  • Member of the Local Administrators group on all OpsMgr role servers (RMS, MS, GW, Reporting)
  • SA (SysAdmin) privileges on the SQL server instances hosting the Operations DB and the Warehouse DB.

These rights (especially the user account having SA priv on the DB instances) are often overlooked.  These are the same rights required to install OpsMgr, and must be granted to apply major hotfixes and upgrades (like RTM>SP1, SP1>R2, etc…)  Most of the time the issue I run into is that the OpsMgr admin logs on with his account which is an OpsMgr Administrator role on the OpsMgr servers, but his DBA’s do not allow him to have SA priv over the DB instances.  This must be granted temporarily to his user account while performing the updates, then can be removed, just like for the initial installation of OpsMgr as documented HERE.  At NO time do your service accounts for MSAA or SDK need SA (SysAdmin) priv to the DB instances…. unless you decide to log in as those accounts to perform an update (which I do not recommend).

 

Ok, Lets get started.

 

1.  Backups.  I run a fresh backup on my OpsDB and Warehouse DB’s – just in case something goes really wrong.  Since I haven’t grabbed my RMS encryption key in a long while – I go ahead and make a backup of that too, just to make sure I have it somewhere.

I also will take a backup of all my unsealed MP’s.   You can do the backup in PowerShell, here is an example which will backup all unsealed MP’s to a folder C:\mpbackup: 

Get-ManagementPack | where {$_.Sealed -eq $false} | export-managementpack -path C:\MPBackup

We need to do this just in case we require restoring the environment for any reason.

 

 

2.  Apply the hotfix to the RMS.

Tip #1:  Here is a tip that I have seen increase the success rate:  Reboot your RMS/RMS nodes before starting the update.  This will free up any locked processes or WMI processes that are no longer working, and reduce the chances of a timeout for a service stopping, file getting updated, etc.

Tip #2:  If you are running any SDK based connectors – it is a good idea to stop these first.  Things like a Remedy product connector service, Alert Update Connector, Exchange Correlation Engine, etc…  This will keep them from throwing errors like crazy when setup bounces the SDK service.

Tip #3:  If you are low on disk space, and you have previously installed prior R2-CU’s, you can uninstall those and make sure they are removed from \Program Files (x86)\System Center 2007 R2 Hotfix Utility\ directory.  This can free up a substantial amount of disk space, and once applied these files are no longer necessary.    

****Note: If applying this update to a RMS cluster – FIRST see:  How to apply a SCOM hotfix to a clustered RMS

****Note:  Many people struggle with OpsMgr hotfixes – for failing to follow instructions.  When applying an OpsMgr hotfix – you need to copy the downloaded MSI file (such as SystemCenterOperationsManager2007-R2CU4-KB2449679-X86-X64-IA64-ENU.MSI) to EACH and EVERY Management server and Gateway.  You need to INSTALL this hotfix installer utility to EACH Management Server and Gateway.  Don’t try and just copy the update MSP files.  This wont work and you will fail to update some components.  Common complaints are that the agents never go into pending actions, or the agent update files never get copied over to the \AgentManagement folders.  In almost ALL cases, people were taking a shortcut and making assumptions.  Don’t.  Copy the 1.26GB file to each machine, then install the hotfix utility, then run the hotfix from the splash screen that comes up, immediately after installing the downloaded MSI.  The only acceptable alternative to this process – is to install/extract the 1.26GB MSI to a workstation, and then build a command line based package as described below.  For memory limited test environments – the command line method is the way to go.

 

Since my RMS is running Server 2008 – I need to open an elevated command prompt to install any SCOM hotfixes. That is just how it is.  So I launch that – and call the MSI I downloaded (SystemCenterOperationsManager2007-R2CU4-KB2449679-X86-X64-IA64-ENU.MSI).  This will install the Hotfix Utility to the default location. 

Tip: (This part may take a LONG TIME to complete if calling the 1.26GB file on a system will limited memory resources.  This is because it must consume 1.26GB of RAM to open the file, temporarily.  For production systems meeting the minimum supported 4GB, this probably wont be as much of an issue.  For virtualized labs and test environments where you are running very limited memory, (1-2GB RAM) you will see this process take a considerable amount of time.  On my 1GB memory virtualized management servers, it would not install at all.  I upped them to 2GB and they took about 10-20 minutes to open and run the setup program.  See section at the end of this article **Command line install** for ideas on how to mitigate this issue if affected)

Eventually – a splash screen comes up:

 

image

I choose Run Server Update, and rock and roll.  You MUST execute the update from this “Run Server Update” UI.  NO OTHER METHOD will work.

 

It runs through with success, I click finish – then another setup kicks off.  This is by design.  There should be three actual setups running consecutively (once for the core update, one for the localization, and one for Xplat.)

You could see this potentially three times:

image

After clicking finish on the 3rd one, I was prompted with this:

image

So – you should plan for a reboot.  Hit OK.  (it wont reboot automatically)

Then wait around 30 seconds for any post install processes to complete, and then click “Exit” on the splash screen.

image

 

If you have trouble at with this stage – get some error messages, or if the installation rolls back – see the troubleshooting and known issues at the KB article and below in this post.  There are two known and fairly common issues encountered with simple resolutions.

If you are patching a clustered RMS – you can continue the process using the link posted above – and complete the second node.

 

 

Now – it is time to validate the update applied correctly.  I can see the following files got updated on the RMS in the standard install path:  \Program Files\System Center Operations Manager 2007\

image

**note – this isn’t all the files included in the hotfix package, just a spot check to make sure they are getting updated. 

 

Next I check my \AgentManagement folder.  This is the folder that any agents will get updates from.  I check the \x86, \AMD64, and \ia64 directories:

image

 

It is good – that our KB2449679 CU4 agent MSI’s got copied over.  In this CU4, we did remove the previous CU files if they existed.

 

***NOTE – it is CRITICAL to perform the next step in this order.  The SQL scripts MUST be deployed at this time, immediately after installing the update on the RMS.  If you don’t, your alert views could be empty.  You could see multiple events on the RMS about errors from the SDK (26319) and DataAccessLayer (33333).  The RMS will not generate new config until these scripts are executed.  Your consoles might also show the following, until you run the SQL scripts:

image

 

 

 

3.  Time to run the SQL scripts.  There are 2 scripts, located on the RMS, in the \Program Files (x86)\System Center 2007 R2 Hotfix Utility\KB2449679\SQLUpdate folder:

  • CU4_Database.sql
  • CU4_DataWarehouse.sql

Let’s start with CU4_Database.sql

I simply need to open this file with SQL management studio – or edit it with notepad – copy the contents – and paste it in a query window that is connected to my Operations (OperationsManager) Database.  I paste the contents of the file in my query window, it takes about a minute to complete in my lab.  It will return a single string of output stating MPLastModified with a timestamp, upon success.

Next up – we now need to connect to the Warehouse database instance, and open a new query window against the OperationsManagerDW database.  We will execute CU4_DataWarehouse.sql which will return “Command(s) completed successfully”.

 

DO NOT skip step number 3 above, and do not continue on until this is completed.

 

 

 

4.  Next up – import the MP updates.  That’s easy enough.  They are located at \Program Files (x86)\System Center 2007 R2 Hotfix Utility\KB2449679\ManagementPacks\ and are named:

  • Microsoft.SystemCenter.DataWarehouse.Report.Library
  • Microsoft.SystemCenter.WebApplication.Library.mp
  • Microsoft.SystemCenter.WSManagement.Library.mp

These will upgrade existing MP’s in your environment.  They take a few minutes each to import.

At this point – if you are using cross platform monitoring for Unix agents – you would upgrade the Xplat MP’s via a separate download.  See the KB article for steps on this, and potentially upgrading your Unix agents if required.

 

5.  Time to apply the hotfix to my management servers.  I have 3 secondary MS servers, one is Windows 2008 and the other two are older, they are running Windows 2003.  So on the 2008 server I open an elevated command prompt to apply the hotfix utility MSI, and just run it directly on the older servers. 

Again – I MUST RUN SystemCenterOperationsManager2007-R2CU4-KB2449679-X86-X64-IA64-ENU.MSI on each Management server.  This installs the hotfix utility, which will then launch the splash screen.

Tip: (This part may take a LONG TIME to complete if calling the 1.2GB file on a system will limited memory resources.  This is because it must consume 1.2GB of RAM to open the file, temporarily.  For production systems meeting the minimum supported 4GB, this probably wont be much of an issue.  For virtualized labs and test environments where you are running very limited memory, you will see this process take a considerable amount of time.  On my 1GB memory virtualized management servers, it would not install.  I upped them to 2GB and they took about 10-20 minutes to open and run the setup program.  See section at the end of this article **Command line install** for ideas on how to mitigate this issue if affected)

Once the splash screen comes up I “Run Server Update”  These all install without issue (again – three setups run consecutively).  I spot check the \AgentManagement directories and the DLL versions, and all look great.  REMEMBER – you can sure patch all your management servers at the same time, however, your agents WILL fail over during this time because we stop the MS HealthService during the update.  Keep this in mind.  It is best to update management servers one at a time, synchronously, to keep your agents from failing over to the RMS and overloading it, or causing massive Heartbeat failures because they have nowhere to report to.

Note:  You might see a “Restart Required” pop up after the last setup routine is complete.  Just ignore this for now and hit OK, and then we will need to reboot this server when we are finished. 

 

6.  Next up – any Gateway machines here.  Since my gateways all have limited memory, I don’t want to run the full 1.2GB MSI.  I am running these from a command line which uses a LOT less resources.  I build a local install package in my local C:\temp\ directory from my article at this LINK using the following command line modified for CU4:

SetupUpdateOM.exe /x86msp:KB2449679-x86.msp /amd64msp:KB2449679-x64.msp /ia64msp:KB2449679-ia64.msp /x86locmsp:KB2449679-x86-ENU.msp /amd64locmsp:KB2449679-x64-ENU.msp /ia64locmsp:KB2449679-ia64-ENU.msp /Agent /noreboot

I “Run Gateway Update” from the splash screen, and setup kicks off.  It runs three separate installs and I see the following – 3 times:

image

Remember to spot check your DLL versions and \AgentManagement directories.   They both should be updated.

 

 

7.  I check my Pending Management view in the Administration pane of the console – and sure enough – all the agents that are set to “Remotely Manageable = Yes” in the console show up here pending an agent update.  I approve all my agents (generally we recommend to patch no more than 200 agents at any given time.)

After the agents update – I need to do a quick spot check to see that they are patched and good – so I use the “Patchlist” column in the HealthService state view to see that.  For creating a “Patchlist” view – see LINK

Out of a total of 41 total agents in my lab – 40 of them patched perfectly.  Only one did not fully patch – it is my Windows 2000 server.  It is possible this older OS still has issues with requiring a reboot between the first update and the ENU localization update MSI.

image

 

The CU4 actually REPLACES any previous patches applied in the PatchList table – this is NICE.   Looks good.  (Note) I will have to formulate a plan to go and update my manually installed agents (Remotely Manageable = No)

Note: See the KB article if your agents will not update.  If you previously applied CU3, or if your agents require a reboot from a previous windows installer package – you cannot update them until they receive a reboot.   Normally an agent side reboot is not required for updating an agent.  It will only be required if it falls into one of these two specialized situations.

 

8.  I have a few dedicated consoles which need updating.  One is a desktop machine and the other is my terminal server which multiple people use to connect to the management group.  So – I kick off the installer – and just choose “Run Server Update” as well.  I do a spot check of the DLL files – and see the following was updated on the terminal server:

image

 

 

9.  Next up – Web Consoles.  I actually have two – and both are running on management servers, which I have already patched.  So – I will simply just go check their DLL files to ensure they got updated:

From:   \Program Files\System Center Operations Manager 2007\Web Console\bin

image

 

 

10.  I don’t have ACS set up at the moment – but at this point if I did – I would go hit those Management servers that have already been patched – but this time run the update and choose to “Run ACS Server Update”

image

 

 

11.  Manually installed agents.  I have a fair bit of these… so I will do this manually, or set up a SCCM package to deploy them.  Most of the time you will have manually installed agents on servers behind firewalls, or when you use AD integration for agent assignment, or when you installed manually on DC’s, or as a troubleshooting step. 

 

Now – the update is complete.

image

The next step is to implement your test plan steps.  You should build a test plan for any time you make a change to your OpsMgr environment.  This might include scanning the event logs on the RMS and all MS for critical and warning events… looking for anything new, or serious.  Testing reporting is working, check the database for any unreasonable growth, run queries to see if anything looks bad from a most common alerts, events, perf, state perspective.  Run a perfmon – and ensure your baselines are steady – and nothing is different on the database, or RMS.  If you utilize any product connectors – make sure they are functioning.

The implementation of a solid test plan is very important to change management.  Please don’t overlook this step.

 

 

 

*** Command line install option 

In some situations, you might want to perform a command line installation of the update on your RMS/management server.  Most of the time – I don’t recommend this, because you generally need the feedback if each part was successful or not.  However, there are situations where it is required. 

One example is for users who have issues with the 1.2GB MSI file, and getting the hotfix installer running, especially on limited memory systems.  For those, you can use a command line options which removes the issue.

The KB article has a section which documents how to set up the arguments correctly.  I used a variation of that, because I did NOT want /silent to be used… as I want to visibly see the feedback and interact with the installation.  Here is the command line I ran, for an US/English installation:

SetupUpdateOM.exe /x86msp:KB2449679-x86.msp /amd64msp:KB2449679-x64.msp /ia64msp:KB2449679-ia64.msp /x86locmsp:KB2449679-x86-ENU.msp /amd64locmsp:KB2449679-x64-ENU.msp /ia64locmsp:KB2449679-ia64-ENU.msp /Agent /noreboot

In order for this to work – you need to INSTALL the hotfix utility somewhere, then copy the ENTIRE FOLDER STRUCTURE starting with the \KB2449679 folder and all files and folders below it.  Here is an example copied to C:\temp\ directory:

image

Just remember – you cannot run just specific MSP files in these folders individually, there are post install processes that must be run, and should be called by the SetupUpdateOM.exe.  You also cannot just run SetupUpdateOM.exe by double-clicking it, or the post install processes wont run.  Use the default method of installing the original downloaded MSI file, OR use this command line option.

 

For additional command line options, including how to make a CU package smaller, and how to patch consoles, agents, etc…. see the following post:

http://blogs.technet.com/b/kevinholman/archive/2010/10/12/command-line-and-software-distribution-patching-scenarios-for-applying-an-opsmgr-cumulative-update.aspx

 

 

Known issues/Troubleshooting:

 

 

1.  Restart of non-Operations Manager services.  In certain cases, non-Operations Manager services may be restarted when the Operations Manager agent is updated. This issue only affects computers that are running Windows Server 2008 or Windows Server 2008 R2. We recommend that you update agents at a time when a service restart or a server restart is acceptable. Or, only update agents that are experiencing one or more of the agent-related issues that are mentioned in the list of resolved issues. This issue will be addressed in an upcoming cumulative update.

When a SCOM 2007 R2 agent is updated to Cumulative Update 3 (CU3) or Cumulative Update 4 (CU4), non-SCOM services may be restarted or a reboot requested. The non-SCOM services can be related to other products like SQL Server, Exchange, Windows, SharePoint, etc.  The service restart may happen on Windows Server 2008 (Vista) and newer, the reboot request may happen on Windows Server 2003 and older.

Cause

A shared code library for logging events has leaked a reference to the dynamic library EventCommon.dll since SCOM 2007 R2 RTM.  EventCommon.dll was first updated in CU3 which is why the service restarts started occurring with that CU.   The performance counter extension MOMConnectorPerformance.dll uses the class library with the leak.  When a process does a snapshot or enumeration of performance counters on a system MOMConnectorPerformance.dll will be loaded into the process doing the snapshot.  The library used by MOMConnectorPerformance.dll will load EventCommon.dll.   After doing a performance counter snapshot Windows will automatically unload MOMConnectorPerformance.dll.  Due to the leak EventCommon.dll is not unloaded and left behind in the process.

While the leak can occur on any version of Windows, the service restart issue only affects Windows Server 2008 and Windows Server 2008 R2. Windows Server 2008 added a feature called Restart Manager which automatically restarts dependent processes when a file those processes depend on is replaced during an update. Windows Server 2003 and Windows Server 2003 may request a reboot after installing the patch.

Workarounds:

1. Plan agent updates for a time when the possibility of services restarts will not impact service level agreements.

2. Update Management Servers and databases with the cumulative update, but wait until CU5 to update the agents.  CU5 will fix the underlying leak.  However, since the update is applying to a system which may have already leaked the reference to EventCommon.dll installing CU5 may exhibit a reboot request as we have disabled the interaction with Restart Manager.  Once a system has been updated to CU5 future updates will not cause a reboot request.

3. You can determine whether a particular computer might be affected by running the following command at the command prompt or in PowerShell and looking for processes besides HealthService.exe & MonitoringHost.exe (specifically WmiPrvSE.exe):

tasklist /m EventCommon.dll

a. For instance, if you have explored the SCOM performance counters via PerfMon, you may find WmiPrvSE.exe (Windows Management Instrumentation service) in the list of tasks that are currently using EventCommon.dll.

b. To determine which services will be restarted if the Windows Management Instrumentation (WMI) service is restarted, run the following command in PowerShell:
(get-service winmgmt).dependentServices

 

2.  CU4 fails to apply.  The SDK or config service may not start after this, and CU4 fails on subsequent retries.  The installation rolls back and you get a dialog box that the setup was interrupted before completion.  There are two possible issues, with workarounds to this.  One is caused by a general timeout, the other is a .NET 2.0 Issue due to a CRL response delay.  Start with workaround “#1” and if that fails, try workaround “#2”.  #2 is a fairly rare condition.

Workaround #1: 

The services are timing out while trying to start.  Using http://support.microsoft.com/kb/922918 set the ServicesPipeTimeout entry for all services to have 3 minutes (180000 milliseconds) and REBOOT the server.  Then try and apply CU4.  It should apply.  You likely will see a few warning messages about failure to start the OMCFG service – just click ok and the setup will continue.

Workaround #2: 

Using Follow the steps that are outlined in Microsoft Knowledge Base article KB936707 

***Note:  This hotfix likely will not be required.  The hotfix is ONLY required if you are still running .NET 2.0 RTM.  This hotfix is included in .NET 2.0SP1 and later.  The hotfix does not resolve the issue, simply put – the hotfix (or .NET 2.0SP1 or later) simply ENABLES the use of a new tag in XML which will allow for disabling of CRL checking.  If your RMS is on Windows Server 2008 or 2008R2 – you already have this hotfix included.

***Note:  Once you have verified you have .NET 2.0 SP1 or later installed – you MUST perform the second step – which involves editing 2 application.exe.config files.  The KB article is misleading in that it tells you to add this information as an entire section – which is incorrect – you must find the <runtime> section in your existing config files – and add a SINGLE new line to that existing section.

The manifest files are located on the RMS at the \Program Files\System Center Operations Manager 2007\ root directory.  The manifest files will need to be edited for the config and sdk service on affected RMS.  The file names are:

  • Microsoft.Mom.Sdk.ServiceHost.exe.config
  • Microsoft.Mom.ConfigServiceHost.exe.config

In between the EXISTING <runtime> and </runtime> lines – you need to ADD a NEW LINE with the following:

<generatePublisherEvidence enabled="false"/>

This solution disables CRL checking for the specified execute-ables, permanently.

 

3.  Agent patchlist information incomplete.  The agent Patchlist is showing parts of CU4, or CU4 but also CU3, CU2 or CU1 or nothing.  The CU4 localization ENU update is not showing in patchlist.  This appears to be related to the agents needing a reboot required by Windows Installer from a previous installation package.  Once they are rebooted, and a repair initiated, the patchlist column looks correct with the CU4 and CU4 ENU (localized) information.  The correct and complete patchlist information will appear as below:

System Center Operations Manager 2007 R2 Cumulative Update 4 (KB2449679); System Center Operations Manager 2007 R2 Cumulative Update 4 (KB2449679) – ENU Components;

 

4.  Authoring Console error when opening a management pack for editing.
When you use the Authoring Console to open a management pack that has been exported from a CU4 applied management group, the authoring console throws an error.  This behavior occurs because the authoring console cannot find the .61 version of the Microsoft.SystemCenter.Library Management Pack.  The reason for this – is that this MP was modified via SQL script update, instead of delivered as a physical file.  This issue is scheduled to be addressed in Cumulative Update 5.  For immediate resolution, there are two workarounds:

A.  Edit the XML of the exported MP.  Modify the manifest section of the MP, and change the .61 to .0. 

B.  Contact Microsoft Customer Support Services for a copy of this management pack

Comments (82)

  1. Kevin Holman says:

    Yes – you can simply import the MP's.

  2. Kevin Holman says:

    @Larry – thats good news….  however – if you see CU3 AND CU4 in patchlist – this is not good.  This means the last part (localization/ENU) update did not apply.  You should see CU4 AND CU4 ENU to represent a fully patched agent.

  3. Kevin Holman says:

    @ JockTodd –

    Agents are placed into pending using the following process:

    When you run a hotfix/CU on a Management server role (including the RMS), there is a step where all applicable agents are bulk inserted into Pending Management (provided the hotfix/CU applies to agents).  This step will place ALL agents into pending following the rules below:

    1.  agents assigned to THAT specific management server ONLY

    2.  Set to Remotely Manageable = Yes

    As long as these two are met, and something didnt break running the hotfix (like failing to follow explicit directions, use elevate CMD, run from original MSI and call "Run Server Update" etc…)  then all agents that meet the criteria above will be placed into pending.

    Agents that will not be placed into pending – will be any manually installed agent, or agents assigned to a different MS/GW that has not had the hotfix/CU applied yet.

    I have heard GW assigned agents are not placed into pending.  I dont know that for sure without further testing, so thatmight be another scenario.

  4. Craig Pero says:

    My earlier comment about disk space for the install has not shown up yet but I found a gateway where I could not get a disk added in a timely manner and could not get the free space I needed for the install.  Here is what I did…

    1) put the CU install file on a server on the same subnet.

    2) opened a command window and did the following in that window

    3) ran the following:     SUBST F: \ServerNamef$

    4) ran the following:     Set Temp=F:

    5) ran the following:     Set Tmp=F:

    6) Changed to the "virtual" F: drive in the command window

    7) Ran the cumulative patch from the command window

    In essense, I did not have the installation locally on the system drive.  when the installer runs, it extracts to F:Temp which is on a remote sever and thus the 2.7 gig of space used between the update and the temp files are on a remote server leaving the minimum files needed on the affected server.

    Lesson learned?  Don't skimp on the system drive!!!

  5. Murad Akram says:

    Kevin, first of all your cat is awesome 🙂 second your posts a very helpful.

    Any idea when SCOM 2007 R2 CU5 is coming out? I am on CU2 and I was wondering if I can avoid one upgrade by going to CU5 straight?

    Thanks

    MA

  6. Kevin Holman says:

    @Max –

    The agent will work fine without the ENU update applied.  We recommend getting it fully patched as soon as time allows, but I dont know of any issues running the agent with only the core CU4 update applied.

  7. Kevin Holman says:

    @Vivak –

    The agent is pushed from the management server it is assigned to.  It doesnt matter where the console is installed.

    The agent version does not change – only the patchlist information – by design.

    If you are missing the ENU portion – go spot check a few agents application log, look for the MSIINSTALLER and RESTARTMANAGER sources logged around the time of the agent update.  I would bet those agents are reporting that windows installer requires a reboot before patching anything else.

    How many agents in this state vs total agents updated?

    What OS version and CPU architecture are these agents?

  8. Anonymous says:

    Hey Kevin.

    Do we really mind spelling checks? Anyway… in this line you probably meant CU4:

    Since my RMS is running Server 2008 – I need to open an elevated command prompt to install any SCOM hotfixes. That is just how it is.  So I launch that – and call the MSI I downloaded (SystemCenterOperationsManager2007-R2CU3-KB2251525-X86-X64-IA64-ENU.MSI).  This will install the Hotfix Utility to the default location.  

    I like your blog as it always saves the day 🙂

  9. Kevin Holman says:

    @Randy – will you email me directly at kevin.holman@microsoft.com ?  I'd like to get some data from you.

  10. Kevin Holman says:

    @Ramesh –

    Shoot!  I just re-read – you DON'T need to run the DiscoveryEntitySprocs.sql script – as that is NOT included in CU4.

    DiscoveryEntitySprocs.sql was included in PREVIOUS CU's as has been integrated into CU4's OpsDB script – CU4_Database.sql

    So you do NOT need to go get it from a previous CU… I gave out the wrong information before… I was going from memory from CU3, and I shouldnt do that.  🙂

    If you did go get it and run it first – it would not hurt anything…. but not required or advised.

    Sorry about the mixup.

  11. Kevin Holman says:

    @Tim – I dont recommend every single hotfix out there… only the ones I have personally dealt with that I found impacting my customers, or heard from enough others that there was significant impact.  I am not a fan of proactively applying any hotfix out there… only the ones which cause impact to your specific environment, or cumulative updates which contain enough fixes that they impact all environments.

    @rob1974 –  CU3 is NOT required.  That is simply stating – that IF you had applied CU3, you already updated the xplat MP's as part of the CU3 steps, and those have not been updated again for CU4.  IF you did not update to CU3, or did not update your Xplat MP's – then you need to do that here as well.

    You can go straight from R2-RTM to CU4 without issue – you just need to update your Xplat MP's from the referenced KB article, and then upgrade your Xplat unix/linux agents per the article.  This is simply saying – that if you did apply CU3, and did update your Xplat environment, no more steps are necessary for CU4.

  12. Kevin Holman says:

    @Edwin –

    Opsmgr does not have a "nag" notification mechanism – it alerts – it notifies – then it expects the customers will resolve or write automation to resolve.  The concept of continuing to email is not a best practice… it simply results in a lot more emails to be ignored.

    However – you can do this yourself – by creating multiple subscriptions of identical criteria – each with a different alert aging (5, 10, 15 minutes) etc….

  13. andyinsdca says:

    Today's ninja trick:

    If you go into Monitoring…Operations Manager..Agent…Agents by Version, you'll note that it's not sortable, making finding agents running an old patch difficult.

    Instead, go into My Workspace, create a new State View and in the "Show Data related to" box, select "Agent," pick a group if you want and your new view will be sortable!

  14. Kevin Holman says:

    Larry – I cannot argue with your logic.

    FYI – the KB article has been updated, and I updated my blog post on the subject.

  15. Kevin Holman says:

    @Ackros – the command line options to kick it off once extracted are covere din the KB article – and in my command line article referenced above.

  16. Kevin Holman says:

    @John –

    It looks like we have identified the issue with restarting multiple OS and 3rd party services when applying the agent update using CU3 or CU4.  The KB article should be updated soon to reflect the information.  This issue does still exist in CU4.  It will be corrected in CU5.  The recommendation will be to only apply agent updates using CU3/CU4 during a change management window where the service restarts can be acceptable.  CU5 should not have this limitation.

  17. Kevin Holman says:

    @cellodom – CU5 is curently tracking towards June/July timeframe.

    @SCOm user – We recommend updating your infrastructure to R2 and the current CU…. not just a few agents.  For the agent upgrade to R2 then a CU, there is very little impact…. the CU3/CU4 service restart is the first issue where we have seen any real impact to an agent.  R2 + CU has similar space constraints to a SP1 agent.  You just get the benefits of all the fixes in the agent code.

    @Oscar – do you think it is related to the known issue at:  support.microsoft.com/…/2425714

    @EdwinM – those overrides are for CLIENT machines only – you generally should not have created those overrides that you did.  Those are in place by design to keep "server down" alerts from being generated.  As to your notification issue – you MUST make sure you include the health service watcher class in your notification subscription scope, and if you filter by group – those groups must contain health service watcher objects.  blogs.technet.com/…/configuring-notifications-to-include-specific-alerts-from-specific-groups-and-classes.aspx

  18. Kevin Holman says:

    Hi Michiel –

    Yes – this has been brought up and is being discussed.  Your workaround is correct and will resolve the issue.  The reason the MP was updated in script is critical – because if customers didnt apply the MP update and SQL script in the correct order, it could cause issues with the database that would require restore from backup.  Therefore it was dont at the same time – in the SQL script – where the order could be controlled.  If we shipped the MP externally as an update, if customers didnt perform the MP update and script in the same order, the results would be MUCH worse than the small pain of changing a reference in the auth console, which any MP author should be completely comfortable with.

  19. Anonymous says:

    Kevin – Thanks for the detailed CU related guides and blog generally….

    Have deployed CU4 across the production environment yesterday and the main outstanding issue is that approx 15% of the previously deployed agents haven't yet appeared in the pending management section for the update/approval option. Restarted OpsMgr services etc and nothing in event viewer or logs etc….any ideas?

    Also, deployed to the Gateway servers and can confirm that the DLL file versions and AgentManagement directories files have been successfully updated in CU4…..  

  20. Kevin Holman says:

    @Dom – CU4 is cumulative – it doesn matter if you have RTM, CU1, CU2, or CU3 installed – you follow the same steps.

    @Tim – I will do that as time permits.

  21. Kevin Holman says:

    @Amaury –

    That will be the topic for my next blog post.  🙂

  22. Kevin Holman says:

    @Sylvester – No – I just find funny pics.  I dont like cats.  🙂

  23. Kevin Holman says:

    @ Tayn –

    This issue was first reported on some Windows 2008 servers in CU3.  It has also been observed in CU4 for some servers, where Restartmanager restarts a large number of services trying to avoice the need for a reboot.  It is being investigated as this is not by design for a SCOM CU.

  24. Kevin Holman says:

    Guys – more information will be coming soon.

    However – I got confirmation on the settings.  Both should be created as a DWORD value.

    "DALInitiateClearPool"=dword:00000001

    "DALInitiateClearPoolSeconds"=dword:0000003c

    The decimal values are:

    DALInitiateClearPool=1

    DALInitiateClearPoolSeconds=60

  25. Kevin Holman says:

    @Michael C – yes this is possible – by calling the correct command line arguments using my command line post examples from CU3, and I believe there is some command line info in the KB article for CU4 now.  Or – rerun the original MSI – and choose "repair".   Not other method can be used.

    @Cyberflake – thanks for that.  Corrected!  As you can see I copied and pasted my CU3 article as a template.  That's good because it makes for a consistent look and feel.  Thats bad because sometimes I miss a few references.  🙂

  26. Kevin Holman says:

    @Vivak – No reboot is required – only prompted if there was something that could not be updated.

    If you PUSH a new agent – it gets CU4 automatically – this is why we doublecheck the AgentMangement directory for the update files.

    If you manually install an agent – you must manually apply the updates.

  27. Kevin Holman says:

    @ Bono – Recommend opening a PSS case.

    @Sam John – this does not include the health service handle count fix – that is an OS hotfix caused by .NET….. you still need to apply it.

  28. Anonymous says:

    Thanks for the walkthrough. I barely had the time to test CU3 in the lab that CU4 comes out…

  29. Kevin Holman says:

    @ Mike –

    The version number does not increment with CU's, by design.  Only the Patchlis value in a healthservice state view as described above.

  30. Anonymous says:

    @Kevin/@OscarG

    Its not only a issue with SQL2008R2 as discribed in KB2425714 . We using SQL2005 and have the same problem. It looks like its a bug in CU4.

    Actually we compare scheduled reports between console an reporting-webinterface. In some reports we found the path to save the reports in the description and the description in the path to save the report, on the  reporting-webinterface, and in the client-console did they look correct. BUT we have to check that issue more detailed, its only a presumtion a the moment.

    Please let me know about your experience.

  31. vah says:

    Thanks again for instructions for CU, keep that way

  32. Amaury Greiner says:

    Hey Kevin, thx for this article.

    I have a question concerning the part

    SOFTWAREMicrosoftMicrosoft Operations Manager3.0DALDALInitiateClearPool = true

    SOFTWAREMicrosoftMicrosoft Operations Manager3.0DALDALInitiateClearPoolSeconds = 60

    of KB2449679. I assume these are registry keys. How do I configure them particularly, are they both DWORD Values? Is the decimal value "1" in this case the equivalent of "true"? I wasn't able to find more detailed information on this point.

    Thx and best regards, Amaury

  33. Dominique says:

    Hello Kevin,

    As always excellent document. As I see CU4 is correcting issues seen in CU3 I wonder as I could not installed CU3 yet if CU4 is an  update of CU3 or could be installed right away?

    Thanks,

    Dom

  34. Tim LJMU says:

    Hi Kevin,

    Please would you do a similar article for the upgrade from SQL 2008 SP1 to SQL 2008 R2 for existing SCOM installations?

    Many Thanks,

    Tim

  35. Awesome! says:

    Thanks for getting this posted so quickly!

  36. Larry Leblanc says:

    First off, your work is much appreciated.  Thanks!

    Secondly, regarding Issue #2:

    – At least with CU3, it was our experience that a Repair sufficed in order to correct the issue (i.e. no reboot required);

    – The patch display order (CU3/4; CU3/4 ENU Components) was not always respected.

    Thanks for

  37. Chris Walker says:

    @Amaury & Kevin –

    I believe this is the configuration for those registry keys:

    [HKEY_LOCAL_MACHINESOFTWAREMicrosoftMicrosoft Operations Manager3.0DAL]

    "DALInitiateClearPoolSeconds"=dword:0000003c

    "DALInitiateClearPool"=dword:00000001

    Kevin…can you confirm…waiting on that next BLOG….   🙂

  38. Michael C says:

    Is it possible to restart the splash screen WITHOUT double-clicking and re-installing the hotfix utility (if already installed), for instance with a command line parameter of SetupUpdateOM.exe?

  39. Toby S says:

    @Amaury, @ JockTodd

    I believe both registry keys should be REG_DWORDs.  REG_SZ will cause the SDK service to crash.

    @Kevin

    I just tried setting DALInitiateClearPool = "true" as a REG_SZ and the "System Center Data Access service terminated unexpectedly" after only a few seconds. The Operations Manager Event Log confirmed it with EventID 26380, OpsMgr SDK Service "…System.InvalidCastException: Specified cast is not valid…".

    It took me a few minutes to realise this was the cause, so you might want to warn about this in your upcoming blog.

    And thanks for your many informative blogs.  I've been finding a lot of useful tips in them.

  40. SN says:

    Thanks for the great writeup Kevin

    Is anyone else having a problem with the /silent parameter? I am using the below command to update my gateway servers, but as soon as I add /silent nothing happens(no errors in my cmd window, no msinstaller entries in app log…just nothing happens)

    SetupUpdateOM.exe /x86msp:KB2449679-x86.msp /amd64msp:KB2449679-x64.msp /ia64msp:KB2449679-ia64.msp /x86locmsp:KB2449679-x86-ENU.msp /amd64locmsp:KB2449679-x64-ENU.msp /ia64locmsp:KB2449679-ia64-ENU.msp /Agent /noreboot /Silent

    I would very much like to use PSEXEC to install across our huge amount of gateways, but this blocks me from doing so

    SN

  41. Tim LJMU says:

    Hi Kevin,

    The Microsoft KB article for CU4 also makes mention of this hotfix :

    The following released SCOM 2007 R2 hotfix is not included in this update because the hotfix includes SQL Server Transact-SQL scripts, SQL Stored Procedure updates, or cross-platform agent updates. For more information about how to obtain a hotfix for this issue, visit the following Microsoft Knowledge Base article:

    981740  (support.microsoft.com/…/981740 ) System Center Operations Manager 2007 R2 does not display new properties in some views after you import a management pack

    I can't find this in your recommended list. Is this something we need to apply?

    Thanks,

    Tim

  42. rob1974 says:

    — CU4 is cumulative – it doesn matter if you have RTM, CU1, CU2, or CU3 installed – you follow the same steps.

    i read in the patch nodes:  If you do not have Cumulative Update 3 for SCOM 2007 R2 installed, you must download the latest management packs for monitoring Linux and UNIX computers before you apply Cumulative Update 4 for SCOM 2007 R2

    i read from this you need to install cu3 before you apply this patch. I think applying cu3 for unix requires cu3 for scom r2 as well i think, so i could skip the linux/unix cu3?

  43. Mick says:

    Hi Kevin,

    just completed the upgrade of cu4 and would now like to upgrade sql 2008 SP1 to R2.  As Tim says could you please put a guide together showing that ugrade path ensuring the SQL instance has  Reporting Services installed as I am getting an error at the Upgrade Rules check stage – No custom security extensions and No Custom Authentication Extensions both fail

    thnaks

    mick

  44. Ackros says:

    I have a clean scom 2007 R2 install on win 2008 R2 with local sql 2008 R2 instance.

    no agents rolled out , zero config done so far.

    was waiting until i got CU4 installed.

    strange thing is I ran the installer, it unpacks into PF (x86) on the (all in one ) server.

    but no menu appears, with the Run XX update..

    trying to figure out how to manually launch it , any ideas?

  45. vivak says:

    After i Installed the CU4 on RMS , I was not prompted for Reboot :(Is this common

    Also If tomorrow i installa new agent using the Discovery , do i have to Install CU4 on that or is it automatic

    If i manually install an agent (i guess i will have to upgrade is) correct

    **Thanks Heaven for Kevin**

  46. vivak says:

    In this case , If i have a RMS and 2 MS servers all of them have the AgentManagemnt dir ,

    How do i know where is the agent being pushed from ..

    The Program Files of RMS always or From the MS where we run the Discovery Wizard From.

    And if in Case i have the console on any other system what then?

    Also when we install the Update I mean approve , does it not chnage the Version on The Agent: In the view:

    Operations Manager..Agent…Agents by Version , I still see the old version , but the patch list colum shows the CU4 One.. Should i be worried.

  47. vivak says:

    Kevin as you mentioned , **You should see CU4 AND CU4 ENU to represent a fully patched agent**

    For some of my agents i just See CU4; they are missing the ENU part in the patch list

    How bad is the state of my agent

  48. Kitaab says:

    I checked the EVentvwr for MSIInstaller on 3 Agents and Yes ofcourse a reboot was required, I rebooted one and repaired the agent and no prize for Guessing " I see ENU Components for it in patch list now)

    For me i updated around 100 Remotely managble = Yes Ones

    and 10 Remotely manageble = No ones  (I used SCCM to Push the Update to them)

    Out of the "Yes" ones 11 Do not show the ENU Components in the patch list  , but now i know the reason Y.

    So over all success in my case for Agents in 91%

    P.S : Can we have more of the Superflows .. SCCM has so many of Those available 🙁

  49. Kitaab = Vivak says:

    Thanks

  50. Sylvester says:

    I love the cat. Is that the same cat as in your R2 CU2 post ?

  51. Tayn says:

    Hi,

    I had a problem on two servers W2K8 R2 when the CU4 applied. Some services are stopped (Server / tcpip netbios / eventlog / dhcpservice / …) ! One for a W2K8R2 server with exchange 2007, and one node of a W2K8R2 cluster.

    Do you have information about this problem? In my case it is therefore arrived on 2 servers knowing that I update about 250.

    It never happened with the CU1 / 2 / 3.

    Thank you in advance, and thank for this blog!

  52. Tayn says:

    Thank for information.

  53. Mike Poole says:

    Hi Kevin,

    I have finished installing CU4 and am now going through my RMS server checking everything over. All seems great but i wanted to check  – When i go to Administration and then Agent Managed, the version of all my agents is 6.1.7221.0 – Should this not be a .61 at the end? All my files have .61 at the end as per your walkthrough however.

    Look forward to hearing from you, thanks again, Mike

  54. Deon says:

    Followed the steps from CU3 to reduce the size of the update for our Gateway servers.  We left off the SCX-Gateway items as we don't do cross platform.  Unfortunately this doesn't work – the 3rd step of the install fails – says it's missing a package.  Once we uploaded the SCX-Gateway folder this went through OK.  We didn't need the SCX files from the root though.

  55. Dahuhn says:

    Facing the same problem you do, Deon.

    Got a "fresh" install of Windows Server 2008 R2, SQL 2008 R2 and Operations Manager 2007 R2 (Trial, downloaded last week)

    Last update step states "missing a package"

    Which folder did you upload ? (Don' t know SCX-Gateway folder existance… did not "start" monitoring but will use cross platform in the future)

  56. Bono says:

    After the upgade to CU4 i cant use the reporting function:

    Cannot Initialize report.

    An internal error occurred on the report server. (rsinternalerror)

    Note:  The following information was gathered when the operation was attempted.  The information may appear cryptic but provides context for the error.  The application will continue to run.

    Microsoft.Reporting.WinForms.ReportServerException: An internal error occurred on the report server. See the error log for more details. (rsInternalError)

      at Microsoft.Reporting.WinForms.ServerReport.GetExecutionInfo()

      at Microsoft.Reporting.WinForms.ServerReport.GetParameters()

      at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.Parameters.ReportParameterBlock.Initialize(ServerReport serverReport)

      at Microsoft.EnterpriseManagement.Mom.Internal.UI.Console.ReportForm.SetReportJob(Object sender, ConsoleJobEventArgs args)

  57. Sam John says:

    Followed the instructions and theCU4 update and it went fine. We had to make the registry changes in our environment. Thanks for providing such detailed guides. Does this update fix the health service handle count threshold error that we get on our RMS. Or do we still need to install a hotfix for it (noticed a hotfix mentioned in some posts?) We are just starting with SCOM 2007 R2 and had our first management pack imported las week. However, the threshold error is there even before pushing agents to the the machines.

  58. Larry Leblanc says:

    As part of our change-management process, we have to detail a rollback plan for the CU4 update. 😛

    Was wondering what rollback strategy is being employed by other OpsMgr admins. Due to the # of interrelated parts being updated, and the sheer complexity of the update, I am tempted to state that there is no failback associated to the change & that we would open a PSS case if need be.

    Your thoughts?

  59. Larry Leblanc says:

    @ Kevin & Tayn –

    We ended up opening a PSS case for the Restartmanager issue during the CU3 deployment, as we had a handfull of Exchange Servers that restarted a slew of services (inc. Exchange) during the agent update.  As evidenced by the blog discussitnow.wordpress.com/…/scom-r2-cu3-agent-update-and-windows-2008, we were not alone dealing with this issue!

    This is what we were told by the engineer handling the issue:

    "Larry,

    We were able to determine that the issue you hit is a repro of a bug that was introduced in CU3 and fixed in CU4 (which just released yesterday).

    Again, I am sorry that you hit this bug and want to assure you that we want to make the CU upgrade process as seamless and risk-free as possible."

    Looks like I'll be re-visiting this issue! 😛

    Larry

  60. Mick says:

    @mick the issue with the upgrade to sql 2008 R2 and the rule check failures – "No Custom Security Extensions" and "No Custom Authentication Extensions" was due to not running the SRS Upgrade tool before the SQL upgrade as explained in an old article explaining the sql 2005 to 2008 process which still works for SQL 2008 to 2008 R2

    technet.microsoft.com/…/dd789004.aspx

  61. Ramesh says:

    Kevin – We've never applied any CU in our environment and still with R2 RTM. For CU4, there is no need to run DiscoveryEntitySprocs.sql (e.g., in our case)? please confirm

  62. Ramesh says:

    Thanks Kevin, but it's not mentioned in this blog, so wanted to double check 😉

  63. Max says:

    Hi Kevin, great read as allways.

    Just a question already asked, if an agent is missing the ENU update, will the agent work or do I have to restart the server and repair thhe agent ASAP?

    I just updated a customers environment and out of the first 4 agents 1 needed a restart. All servers are running Windows 2003.

    Regards

    Max

  64. Max says:

    Just noted that I tried to restart the health service and then repair the agent. Now I can see the ENU update in the patchlist without restarting the server. I hope its working as it should 🙂

  65. randy says:

    Those of you running Windows 2008 are rolling the dice on this update making RestartManager pretty much tank your server by restarting all manner of core services.

    Just had it happen here in Dev and I'm seeing it complained about elsewhere and that it goes back to CU3.

  66. Michiel Wouters says:

    Kevin,

    After applying CU4, Microsoft.SystemCenter.Library management pack is updated internally to 6.1.7221.61.

    When using the authoring console on MP's created in a CU4 Management Group, you are prompted for Microsoft.SystemCenter.Library.mp 6.1.7221.61

    However, the MP file is not available from the CU4 installation sources. I blogged about it on my blog. michielw.blogspot.com/…/scom-cu4-and-microsoftsystemcenterlibra.html. I hope Microsoft releases the .mp file soon.

  67. John Gandee says:

    We just ran across the RestartManager issue with CU3 when trying to re-install and agent.  (the core applications restarted, but unlike other posts I have seen this was not a Microsoft Application server (OCS, Exchange etc.)).    We want to move forward with CU4 but it does not seem that this issue is resolved in CU4.   Are there any configuration workarounds that can be applied against RestartManager?

  68. Larry Leblanc says:

    1 – When we encountered the RestartManager issue in CU3, Microsoft also told us that the issue would be corrected in CU4.  Considering the fact that we have experience the same problem in CU4, I would strongly recommend that CU5 be deployed in a change management window, despite the fact that we have also been told that the issue would be resolved in CU5.  Also, though Microsoft has provided a workaround which involves the implementation of a GPO to disable RestartManager, they do not recommend its implementation, as it would apply to all MSI installations which may have other unknown effects.

    2 – We also encountered an issue which appears to break UAC on certain systems.  Microsoft has confirmed this We are beginning to noticed a correlation with this issue, and the installation of the Microsoft Visual C++ 2008 Redistributable

    3 – Seeing as I have not received a response to this inquiry (above), it's unfortunate that Microsoft does not include a comprehensive rollback strategy when it comes to the deployment of the CU. 🙁

  69. Craig Pero says:

    Space issues for the install are a pain on some of our gateways which are virtual machines with system drives that are too small and free space could not be made even when we added a second drive for the cumulative Update file.  With the update file on a different drive, I got back more space by opening a CMD window and manually setting the TMP and TEMP environment variables to our second drive.  then I ran the update from the CMD window (no double clicking in explorer) and the 1.2gig temp extract went to our second disk which gave me the space needed to run the installation.

    Just in case someone is looking for ways to get that last bit of space on the DISK COST dialog.

  70. cellodom says:

    Hi Kevin

    I've seen that you recommend to wait for CU5, 'cause there are some known issues with CU4 (services under W2k8/R2). I have some questions about it: a) Is there already a release date for CU5? Since the problem has existed already for CU3, we have not noticed that mistake before. We install first the agent, manually stop the health service and install then hotfixes. With this approach, we could not detect the error. Is this maybe a workaround?

  71. SCOm user says:

    Hi All,

    im badly in need of a help…Currently we are using SCOM 2007 SP1 in our environment. We have a taken a step ahead and decided to upgrade only 4 Dc Machines consisting of SCOM agent to SCOM 2007 R2  along with Cu4. All the four DC's are of WINDOWS sERVER 2003 R2 machines.

    Now my question is —- Any effects in directly upgrading them to R2,

    What precautions must be taken before the change implementation

    Wat about the space constraints ?

    And anybody has any procedure to directly upgrade the DC from SP1 to R2.

    Please reply to this post ASAP…

    Thanks in Advance

  72. OscarG says:

    I have installed CU4 in my SCOM env, this is 4 MG and 1 shared DW, and I have migrate SQL to SQL 2008 R2 and everything is working fine, juts I found an issue today.  I can't open the Schedule Reports in the Operations Console I got the folloing error.

    Date: 3/31/2011 11:22:53 AM

    Application: System Center Operations Manager 2007 R2

    Application Version: 6.1.7221.61

    Severity: Error

    Message: Report subscription list could not be loaded.

    System.IndexOutOfRangeException: Index was outside the bounds of the array.

      at System.Array.InternalGetReference(Void* elemRef, Int32 rank, Int32* pIndices)

      at System.Array.GetValue(Int32 index)

      at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.ManagementGroupReporting.GetSubscriptions(String owner)

      at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.Views.ReportSubscriptionsView.ReportSubscriptionsView.LoadSubscriptionsJob(Object sender, ConsoleJobEventArgs args)

    Any clue about what is the cause of this issue?

    I can open the URL http://reportserver/Reports fine and without issues, just when I tryied to open the schedule report in the operations console I got the mentioned error.

    Any help is appreaciated.

    Thanks

  73. cellodom says:

    OscarG

    We have the same issue since we installed CU4. The only workarund we found is to install the console on a client (OS) like vista. Opening the scheduled reports from the client-console is working. Please let me know about your experience.

  74. EdwinM says:

    Thanks for the great read!  I just did a clean install of SCOM 2007 R2 yesterday then did the CU4 update as advised.  I noticed that by default, Health Service Heartbeat Failure and Computer Not Reachable, have the Generates Alert parameter to false.  I have created two overrides and I receive alerts but do not receive a notification email.  I know my notification channels are set up correctly because I receive emails from other alerts, just not these two.  Any thoughts?

  75. EdwinM says:

    Thank you so much for your quick response!  I realized after I had sent the previous post that the override was meant for client machines.  Currently I have an email subscription to notify on all Critical and New alerts (I added the New clause so I don't receive an email when I close alerts) and have alert aging to send notifications without delay. I do receive an initial email now when a critical alert occurs but nothing else after that.  I am expecting to keep receiving emails as long as the alerts are in a critical and new state. If a critical alert occurs, I would like to keep receiving emails until its either acknowledged or resolved.  Any advise would be greatly appreciated!

  76. Larry Leblanc says:

    FYI, regarding the inclusion of a comprehensive CU rollback strategy, it was mentioned at MMS 2011 that this functionality would not be included before the CU7-CU8 timeframe.

  77. Ramazan (MVP) says:

    After installing CU4 I don't see any alerts anymore in the admin console, anyone seen this?

    Thanks for your support!

    Regards

    Ramazan

    ramazancan.wordpress.com

  78. ChrisT8T says:

    Post CU4: We are fighting an issue where some servers (2003/2008) are tanked when we start the Health service. After a period of time they become unresponsive.

    Grey Screen/Black Screen at login. Still pingable rom an OS stand point but tanked enough to cause the applications to fail.

    One of our DC's (FSMO Role Holder) was tanked; took down several other applications that relied on querying a GC.

    Rather than troublshoot the issue,upper management decided to disable the SCOM services on all production servers.

    It required a complete uninstall of the Agent and then a Re-install.

    Fixed the issue but still makes me nervous about restarting the agents on machines that have known issues with this update.

    This requires me to go either go and upgrade them manually now or uninstall / reinstall.

  79. justin says:

    Kevin,

    Should a restart be performed between steps 2 and 3?

    Thx

  80. Version number incorrect at Gateway Registry?! says:

    We saw several installations where our Gateway Servers were updated correctly (DLL went to .61 version no.) but within the Registry the version is still at .0 ??!! Bug?!

  81. Phil McIlwraith says:

    @Oscar/@cellodom/@Kevin,

    The scheduled reports folder problem is a confirmed bug with CU4.  The workaround is to browse to http://<reporservername>/reports and select "My Subscriptions" (top right).

    Microsoft support stated that this problem will be corrected in CU5.

    Cheers

  82. adam says:

    In the MS KB article it says if you don't have CU3 installed, to install the latest cross-platform monitoring mp's first. The instructions for installing the latest cross-platform management pack say to install CU3 first, re-discover unix and linux systems, upgrade those clients and then install the mp's. I do not monitor any unix or linux systems currently but plan to later. Can i just import the latest cross-platfforms mp's into SCOM 2007 R2 with no CU's applied and then upgrade right to CU4 instead.