OpsMgr 2007: Monitoringhost.exe or HealthService.exe may have a high (>10000) handle count and restart


image


UPDATED 08/17/2009: This issue has been resolved with Knowledge Base article 968760: http://support.microsoft.com/kb/968760.


======== 


Here’s kind of an interesting issue I thought we should probably give you a heads up on.  It’s not really a ‘problem’ per se but it is something that can cause some concern if you don’t know what’s going on.  Anyway, here you go:


========


Issue: On Windows computers acting as a Management Servers or agents for System Center Operations Manager 2007, Monitoringhost.exe or HealthService.exe may have a high (>10000) handle count and restart. This issue manifests primarily on 64-bit Windows systems, but we have also had a few reports of problems on x86 systems.


You’ll usually notice this because the ‘Health Service Handle Count Threshold Exceeded’ monitor goes critical and the healthservice is restarted, or the ‘MonitoringHost Handle Count Threshold’ rule fires, generating an alert and restarting Monitoringhoste.exe. Once the healthservice is restarted, the count returns to normal but will begin increasing as the system works.


Using the handles.exe tool from Sysinternals to dump the affected process shows a very high ‘Events’ handle count and may also show a large number of thread handles. Following is a sample output from an affected system.


Handle v3.42
Copyright (C) 1997-2008 Mark Russinovich
Sysinternals – www.sysinternals.com
Handle type summary:
ALPC Port : 11
Desktop : 1
Directory : 2
EtwRegistration : 36
Event : 16010
File : 452
IoCompletion : 23
Key : 48
KeyedEvent : 2
Mutant : 9
Section : 15
Semaphore : 82
Thread : 3956
Timer : 5
Token : 183
TpWorkerFactory : 2
WindowStation : 2
Total handles: 20839


Cause:  This occurs because the Garbage Collector won’t run until enough objects have been allocated. In this case very few (sometimes zero) objects are being allocated. The runtime tracks the threads with native data structures but relies on finalization to clean them up. Since the garbage collection never runs, finalization never runs, and as a result the native data handles are never cleaned up.


Workaround: The high handle count does not appear to cause any performance issues – it is problematic because the SCOM monitor will go critical, and also because it can theoretically continue to grow with no practical upper limit, effectively rendering the monitor useless for health state monitoring.  The upper limits for the ‘Health Service Handle Count Threshold Exceeded’ monitor and the ‘MonitoringHost Handle Count Threshold (Management Server)’ and ‘Monitoring Host Handle Count Threshold’ rules can be increased. The upper limit of this can be overridden to 50,000-100,000. This does mean the healthservice and/or monitoringhost.exe will eventually be restarted but increasing the limit allows more time between restarts.


Hope this helps!


J.C. Hornbeck | Manageability Knowledge Engineer

Comments (3)

  1. Anonymous says:

    Feed: The Operations Manager Support Team Blog Posted on: Thursday, April 02, 2009 6:45 PM Author: jchornbe

  2. jason says:

    I'm seeing this on a windows server 2003 machine but the page doesn't seem to have a fix for windows server 2003, just 2008.

  3. show box says:

    Thanks for the great info. I really loved this. I would like to apprentice at the same time as you amend your web site, how could i subscribe for a blog site?
    For more info on showbox please refer below sites:
    http://showboxandroids.com/showbox-apk/
    http://showboxappandroid.com/
    Latest version of Showbox App download for all android smart phones and tablets.
    http://movieboxappdownloads.com/ – It’s just 2 MB file you can easily get it on your android device without much trouble. Showbox app was well designed application for android to watch movies and TV shows, Cartoons and many more such things on your smartphone.
    For showbox on iOS (iPhone/iPad), please read below articles:
    http://showboxappk.com/showbox-for-ipad-download/
    http://showboxappk.com/showbox-for-iphone/
    Showbox for PC articles:
    http://showboxandroids.com/showbox-for-pc/
    http://showboxappandroid.com/showbox-for-pc-download/
    http://showboxforpcs.com/
    There are countless for PC clients as it is essentially easy to understand, simple to introduce, gives continuous administration, effectively reasonable. it is accessible at completely free of expense i.e., there will be no establishment charges and after establishment
    it doesn’t charge cash for watching films and recordings.
    http://www.showboxforipad.org/showbox-apk/ Not simply watching, it likewise offers alternative to download recordings and motion pictures. The accompanying are the strides that are to be taken after to introduce Showbox application on Android. The above
    all else thing to be done is, go to the Security Settings on your Android telephone, Scroll down and tap on ‘Obscure sources’.
    http://www.showboxforipad.org/
    http://movieboxappdownloads.com/moviebox-apk-android/
    http://movieboxappdownloads.com/download-moviebox-pc/
    Movie Box, an esteemed movies application in which you can find stacks of programs and films. The guide is given here to download Movie Box app to Android and to Apple iOS 9.0.2, iOS 8.4/8.3 and also for the lower versions without Jailbreak.
    http://showboxforiphone.org/
    Please do login to Showbox application with the help of Ymail. You can login in Ymail from here –
    http://ymaillogintips.com/
    Sign Up & Do registration for latest movies on Showbox application