The Case of the Slooooow System


A few weeks ago my wife complained that her Vista desktop was not responding to her typing or mouse clicks. Given the importance of the customer, I immediately sat down at the system to troubleshoot.  It wasn’t completely hung, but extremely sluggish. For example, the mouse moved and when I clicked on the start button the start menu opened after about 30 seconds. I suspected that something was hogging the CPU and likely could have resolved the problem simply by logging off or rebooting, but knew that if I didn’t determine the root cause and address it, she’d likely be calling on my technical support services again in the near future. In any case, stooping to that kind of troubleshooting hack is beneath my dignity. I therefore set out to investigate.


My first step was to run Process Explorer to see which process was using the CPU. After a few minutes Process Explorer finally appeared and showed that not one, but two processes were involved, each consuming 50% of the CPU: Iexplore.exe and Dllhost.exe. Iexplore is Internet Explorer (IE) and I suspected that IE itself wasn’t the problem, but that it was a browser helper object (BHO), ActiveX control, or some other plugin loaded into IE. Similarly, Dllhost.exe is the host process for out-of-process COM server DLLs, so it was probably not at fault, but the COM server loaded into it. Both required digging deeper and I decided to tackle IE first.


In order to try and get some CPU headroom in which to operate, I suspended the Dllhost process by selecting it in Process Explorer, right-clicking to open the process context menu, and selecting the Suspend entry:


image 


That put the Dllhost process to sleep and, as I expected, that freed up 50% of the CPU. That’s because the computer was a dual-core system and so to consume 100% of the available CPU cycles a process would have to have two threads, each hogging one of the cores. Most bugs I’ve seen that result in the CPU being pegged are caused by a single thread.


Processes don’t execute code, threads do, so I needed to look inside the IE process to see what thread or threads were running. I double-clicked on Iexplore.exe in Process Explorer to open its process properties dialog and switched to the Threads page. Several threads were running, but one was dominating the CPU:


image 


From past experience I knew that Ieframe.dll was part of IE, but to be sure I clicked on the modules button on the Threads tab of the Properties dialog and switched to the Details page of the resulting Shell properties dialog:


image 


The description didn’t give me a clue as the thread’s specific purpose, so I moved to the second clue about the thread, its start function. Because I had configured Process Explorer to retrieve symbols for Windows images from the Microsoft symbol server in Options->Configure Symbols, Process Explorer showed the name of the function where each thread began executing. Sometimes the DLL or function where a thread starts executing is enough to identify the thread’s purpose or the software causing a problem. In this case, the thread began in a function named CTablWindow::_TabWindowThreadProc. The function name hints that it’s the one in which the main thread of a tab starts running, but that still wasn’t enough to tell me why the thread was running so much; I needed to dig even deeper and look inside the thread to see where it was executing.


To look at what the thread was up to, I double-clicked on it in the Threads list to open the Thread Stack dialog, which shows the functions on the thread’s stack. A stack is essentially an execution history, where each function listed called the one above it on the list and the function at the top of the list is the one most recently executed by the thread at the time of Process Explorer looks at the stack. I scrolled through the list, looking for frames that referenced 3rd-party DLLs or Microsoft IE plugins, since they would be far more likely to have a bug than IE’s own code. Sure enough, I found frames pointing at a popular 3rd-party ActiveX control, Adobe Flash:


image


Just to be sure that I hadn’t happened to catch Flash running when a different component was using most of the CPU time, I closed and reopened the stack dialog several times, but all of them pointed at Flash.


The first thing I do when I suspect that some software is causing a problem is to check the vendor’s web site to make sure that I have the latest version. I opened the Process Explorer DLL view and looked at Flash.ocx’s version, went to Adobe’s site and looked at the version of the current Flash download, and they were the same. 


I was at a dead end. I couldn’t know for sure if Flash had a bug or, more likely, there was a Flash application that had a bug, nor could I be sure that the problem wouldn’t recur. I tried to determine which site was hosting the Flash content by closing tabs one by one, but when I had close them all the thread was still running.


At this point the only options I had were to uninstall Flash and leave my wife with a degraded web experience, or terminate IE to stop the current CPU usage and hope that it wouldn’t happen again. I chose the latter and the case remains open. Since investigating this I’ve seen the same Flash behavior again on my wife’s system and on my own, so have been vigilantly watching the Adobe site for a new version just in case its due to a bug in Flash itself. I was disappointed that there was no actionable result of the investigation, but at least I knew what had caused the CPU usage.


I now turned my attention the Dllhost problem with the hope that I’d meet with better success. Process Explorer lists in a tooltip the component or components loaded into hosting processes like Svchost.exe (the Windows service host process), Rundll32 (the Control Panel applet hosting process), Taskeng.exe (the scheduled task hosting process on Vista and Server 2008), and Dllhost.exe. I moved the mouse over Dllhost.exe to see what COM server it was running:


image


It was running the Thumbnail Cache COM server, whose job it is to create Explorer thumbnails for image and media files. It is part of Windows, so once again I had to look inside the process for more clues. I resumed the Dllhost process I had suspended earlier and opened the process properties threads page:


image 


The thread consuming the most CPU in this case started in Quartz.dll’s ObjectThread function. I looked at its properties and saw that it was another Windows DLL, the DirectShow Runtime, with a generic function name:


image


Next, I double-clicked to look at the thread stack:


image


The first few frames were in User32.dll and Ntdll.dll, core Windows system DLLs, but frames 4-7 are in the Sonicmp4demux.ax (“.ax” is an extension commonly used for DirectShow filters), a 3rd-party component. The function names for those frames were the same and didn’t make sense because the Microsoft symbol server only stores symbols for software included in Windows. Several more stack snapshots confirmed that it was the code causing the CPU usage.


Now that I had my suspect, the next step was to check for a newer version. But first I had to figure out what software the DLL came with, which was harder than it seemed. I opened the DLL view to take a closer look at the version information, but the description didn’t reveal anything:


image


There were no folders in the Start menu or items in the Add/Remove Programs list with Sonic in the name. I Windows-Live-searched (I expect that word to be added to Webster’s any day now) for Sonic and found that it’s part of the Roxio’s CD and DVD authoring software suites. I looked in the start menu and sure enough, found a Roxio folder:


image


I ran the Roxio software to check its version number and discovered that the Creator application includes a built-in facility to check for updates. I ran it, but it came up empty:


image


I checked the Roxio web site just to be sure and it turned out there was a newer version that the built-in updater hadn’t offered, perhaps because the update, according to the page, didn’t offer anything new:


image


I downloaded it anyway (all 640MB of it!) and waited the 15 or so minutes for it to install. Then I checked the version information of Sonicmp4demux.ax to see if it was newer, but its version number, 1.4.402.60802, was the same as the one I’d seen in the DLL view and the file was two years old:


image


I could have uninstalled the software, which would ensure that the problem wouldn’t return, but I wanted to keep Roxio for its DVD authoring functionality. I didn’t care if I didn’t get thumbnails for Roxio-specific image formats – I wasn’t even sure there were any I’d ever see in Explorer – so I set out to see if I could disable just the Sonic demultiplexer. I could have searched the Registry for the DLL name, which is surely where it was registered, but that’s a brute-force approach and if there were indirect or multiple references I could easily end up disabling more than just its thumbnail generation and possibly breaking something in Windows.


Process Monitor was the perfect tool for the job. Because I didn’t know when the problem might reoccur – it might takes days to reproduce – I didn’t want to just run it and let it consume all available virtual memory or disk space, so I set the History Depth in the Options menu to have Process Monitor retain only the most recent 1 million events:


image


I also set an Include filter for paths matching C:\Windows\System32\Dllhost.exe, minimized it, and let my wife have the system back.


The next day I came home from work, sat down at the computer and saw from Process Explorer that Dllhost.exe was back at it, consuming 50% of the CPU. I suspect that because it’s a dual-core system, the problem had been showing up regularly, but my wife hadn’t noticed it because the remaining CPU capacity was enough to mask it (another good reason to buy multi-core processors!). I brought Process Monitor to the foreground and noted it had seen 114,000 Dllhost operations, which was obviously way too many to scan through individually. I searched for “sonicmp4” and found a reference in a Registry query near the end of the trace:


image


The query is of a COM object registration for the demultiplexer. Because the COM object is a 3rd-party DLL, I was certain that that COM Class ID (CLSID) isn’t hard-coded into Windows, so I went back to the first entry in the trace and searched for “A7DD215”, the first few characters of the CLSID. The search found a match a few thousand operations earlier:


image


The CLSID was in the name of a Registry key under another COM object registration. I Windows-Live-searched (that just rolls off the tongue, doesn’t it?) for the parent CLSID and found this KB article that explains that the registry key is where DirectShow filters register: http://msdn.microsoft.com/en-us/library/ms787560(VS.85).aspx  I took a look at the stack for the particular query to confirm that’s the reason Dllhost was reading from there:


image


I was now confident that I could simply rename the Sonic filter registration key to prevent its use. I never delete registry keys when performing this kind of troubleshooting just in case the change disables important functionality or somehow breaks something else. I had seen from the traces that the thumbnail cache generator had come across an AVI file that caused it to load the Sonic demultiplexer, a format Windows is obviously able to handle on its own, so I was pretty sure things would continue to work. After terminating the Dllhost and making the change, I browsed to the same folder, deleted the thumbnails, and confirmed that there was no reduced functionality as far as I could tell. I then used Roxio to successfully burn a DVD with a number of AVI files. This case was closed.


My wife’s system was now usable again, and though I wasn’t able to close the Flash-related part of the case, at least I knew the cause and could keep an eye out for updates. More importantly, by solving the Dllhost part of the case, even if Flash went crazy again, her system would still be usable and she wouldn’t be filing a critical support incident for it with me – thanks to Process Explorer and Process Monitor.


Comments (156)

  1. Anonymous says:

    You're a lifesaver ! I had the same problem, but you really helped me. thank you !

  2. Anonymous says:

    @Pentti Hirvo

    Nothing in this case was Vista-specific.

  3. Anonymous says:

    Doc wrote:

    "Considering all that? I’m sorry Mark. Vista should be good enough to prevent this from occurring."

    Rune wrote:

    "_How_?

    Until you can answer that simple question, you are just a troll."

    Build in to the OS that what a few 3th Party tools already are doing: Automatically and temporally lower the priority of any user process that is causing a High CPU load for more than x seconds, where the time the priority is lowered and the thresholds (% CPU load and duration) are both configurable,

    and there also should be a way to exempt processes from this.

    There are even tools that do this also for memory , where you can assign a maximum memory limit to each user (in a multi user environment)

    It is quite common to use tools like this on multi user environments like Citrix XenApp servers to prevent the users perceived performance to become unacceptable  when some user starts a high CPU or memory demanding application, you think flash is bad, you should see what some Business reporting software can do to a XenApp server with 30 – 50 users if you don’t use tools like this

    – Rob

  4. Anonymous says:

    Mark, you’re too kind. I would have kicked Roxio off my systems for good soon as I realized I need to hack the registry to fix things.

  5. Anonymous says:

    The reason that the system became unusable in this situation is due to the fundamental way that the NT scheduler has worked since NT4, for good or for bad. You can read the gory details in my book "Windows Internals", but I’ll give a brief explaination here.

    By default, threads run at priority 8, but get priority boosts for short periods when they wake up from sleeping or waiting and when they get a windows message and they are their window is the foreground.

    What happened on my system was that the flash and dllhost threads had priorities set to "above normal", which is 9 for threads of processes running in the default process priority class.

    When I unlocked the screen, clicked on the start menu, and interacted with other processes running at the default priority of 8, their main window thread got a short boost to 12 and then got dropped back to 8. At that point it was below the priorities of the active threads. Windows includes an anti-starvation priority boost that kicks in every several seconds and temporarily boosts the starved thread’s priority to 15.

    The long delay to see a response to input was therefore the result of the window’s thread finally getting enough of those short priority 15 boosts to display it fully. In the case of the start menu, that was around 30 seconds.

    Yes, it’s not great, but it’s been that way for 13 years across many OS releases – NT4, Win2K, XP, Server 2003, Vista and Server 2008 – and you’ll only see this kind of problem in special circumstances – like I did. That said, we’re always looking at things like this to see how we can improve it while keeping in mind the problems of application compatibility and avoiding unintendend side effects.

  6. Anonymous says:

    @Gabriel

    >Do you know why these threads were running with priority "Above Normal."  

    No, but it’s not uncommon for programmers to program their threads with higher priorities with the intention of making their operation more responsive. In most cases programmers should leave their threads at the default priority.

    Yes, Process Explorer sets its priority to "High" so that it can operate even when threads like the ones in this case are in CPU-bound loops.

  7. Michael Dragone says:

    I experience the same behavior with Flash frequently.

  8. Sarat says:

    You’re Insanely smart as Larry Osterman said and dear hubby :D.

    I’m also digging into a sluggish performance problem with my nVdia 8800 card. It’s using the latest Display Driver available for Windows XP. The hardware seems to be working fine with Vista (with it’s default driver).

    Anyway thanks a lot for your posts on trouble shooting and debugging.

  9. Jonathan says:

    Re. Flash: It could be because some lame Flash programmer wrote a polling loop in ActiveScript. No amount of version updates will get you out of it.

    Re. sonicMP4Demux.dll: I presume you could just run "regsvr32 /u sonicMP4Demux.dll", and have it deregistered.

  10. fds says:

    You should be able to unregister any DirectShow filter using regsvr32 /u.

    regsvr32 /u Sonicmp4demux.ax

    The name sounds like it’s a demuxer for .MP4 format files, which is basically the same as Apple’s QuickTime .mov format.

  11. JaredPar says:

    "I Windows-Live-searched (that just rolls off the tongue, doesn’t it?)"

    I prefer to call in learching.  Since the live team didn’t choose a name which could be used as a verb (aka google-ing) we need to get creative.  Thus comes learching: Live sEARCH-ing.  I prenounce (probably incorrectly) as lurching.  

  12. Ehsan Samani says:

    Mark Why is it that this flash thing does not suprise me?

    Flash always hangs my machine. causes my IE to crash and etc. thanks god IE8 has tab recovery feature.

  13. Joe Butler says:

    Given the importance of the customer… Ha!

    Anyway, did anyone else do a double-take during the opening credits of the new Smallville.  

    Does Mark have just a passing resemblance to the Green Arrow?  

    http://thetvaddict.com/2006/09/22/smallville-scoop-2/

    Will read your article now…

  14. Lionel says:

    Would it be possible to change the thread scheduling algorithm in order to improve the system’s responsiveness in such circumstances (high cpu load)?

  15. Gerard Leenhouts says:

    I was waiting for one of these articles to appear again!  It was as entertaining and educational as always.  Thanks a lot, Mark.

  16. Kevin says:

    I was having alot of issues with high cpu loads on my core2duo system, and whenever i would look deeper into the problem that was causing it, it was almost always related to either a audio/video application hanging, or related to the "Multimedia Class Scheduler" service, thats new with vista, and came with SP1 i believe.

    The biggest issues i seem to have are all related to the fact that i typically have ALOT of Directshow filters and plugins installed at any one time, because i do alot of video production, and have to take video from alot of different formats, and hack them to pieces and put them back together.

    Its mostly due to the fact that i have to import alot of video that is not native to the program i am using, or I have to use a media splitter/demuxer to fool the application into importing the video at all.

    I pretty much solved all my CPU hogging issues by doing this –

    uninstall quicktime and anything that relies on quicktime, such as itunes

    install K-lite codec package

    install Quicktime Alternative

    install the Quicktime-itunes addon, so that you can still run itunes and any other cranky quicktime dependant application without having quicktime actually installed.

    you can get all those programs from http://www.codecguide.com

    it is an absolute godsend for anyone who is needing to edit, process, or format video for many different devices, and seems to help alot with keeping media apps that are dependant on media splitting, demuxing, or importing through directshow from hanging and just burning up cpu cycles

  17. Phileosophos says:

    So the real question, or at least the one I find most interesting, is this: when will the various bits of Windows/IE that so glibly host third-party code detect and reign in such excesses?

  18. Walt says:

    Great article Mark! What’s the average customer to do however? Inevitably they will take their system back to where they bought it, the technicians won’t find the problem but will reinstall the operating system. Everything will be fine until the customer re-installs the offending software and then the whole thing starts all over again.

  19. Tyler says:

    Mark,

    Thanks for some insight on how to utilize Process Explorer better.  I was always frustrated with finding out what addin causes IE to crash.

  20. Nick says:

    Mark, I think this case is interesting because would really really like to understand why those 2 processes were able to make the entire system almost unusable.  Can you shed any light on the following questions:

    Why does it take 30 seconds to open the start menu ?  If there are 2 processes running, each of them taking up almost 100% of the CPU cycles of one core, why is the Windows scheduler not able to allocate some reasonable fraction of the CPU cycles to other processes, especially Explorer.exe ?

    It seems utterly bizarre to me that one process running in a tight loop can make the whole machine almost unusable.  Is it not reasonable to expect the scheduler to evenly split the CPU between processes or threads ?

    I remember using a Sun-3 machine (Motorola 68020, running some version of Sun’s Unix) back in 1989 and if there was a runaway process (or two), everything would slow down proportionally.  So if you had 2 processes spinning in a tight loop, and you wanted to do something else, that "something else" would get 33% of the CPU cycles.  That seems much better than giving almost all CPU cycles to those 2 processes and almost none to Explorer, so that the start menu takes 30 seconds to respond to a click.

    I would really love to find out why Windows allocates the CPU cycles the way it does.

  21. Stephen Clarke-Willson says:

    My wife had the same problem (the Flash problem).  I switched her to Chrome because the Chrome task manager makes it very easy for her to identify a page with a wacky out of control Flash widget and kill it, or, alternatively, kill Flash.

    This kind of problem is only going to increase as random people write poorly tested scripts that go into infinite loops.  *Sigh*.

  22. Doc O. Backbush says:

    As an IT tech, I’ve encountered this behavior from windows quite frequently. Third party applications acting haywire. That was half-expected on 98, ME, even XP. But…

    Considering how long Vista was in development…

    Considering the cost of the OS…

    Considering the intended audience that this flash-bang gee-whizz OS was created for…

    Considering all that? I’m sorry Mark. Vista should be good enough to prevent this from occurring. Vista should be able to manage this so the end-user…a home customer…could manage any such problems without being married to one of the leading software debugging techs in the world.

    Thank you for a very informative blog entry. I’m sure it’ll be shared with anyone coming to be for recommendations on Windows Vista.

  23. Steven says:

    I have experienced similar slowness for the past year or more. However, my diagnostics skills do not match yours.

    I finally gave up and got a Mac. After 18 years of Windows, I was ready for a change. I’m glad I did. It’s been a long time since I was this happy with a computer. Sure I’ve had a few issues with my Mac, but far fewer in the months since I’ve bought it than I get in one week on Windows.

    Sorry to sound like a Mac fanboy. I had no intention of leaving Windows…until it left me (with a crappy system).

  24. Vektor says:

    > So if you had 2 processes spinning in a tight loop, and you wanted to do something else, that "something else" would get 33% of the CPU cycles.  That seems much better than giving almost all CPU cycles to those 2 processes and almost none to Explorer, so that the start menu takes 30 seconds to respond to a click.

    TimeSlice of Windows 9X was reasonable. Also Explorer.exe was faster – it didn’t do all the useless stuff the newest versions do.

  25. gary says:

    i used process explorer to track a client’s dual core performance problem, too. it was taking 2 minutes to run an excel app i worte that runs in less than 40 seconds on any other pc they had. fired up process explorer, ran the code and what showed up thousands of times? google desktop search. uninstalled it and the code ran in about 30 seconds. thanks.

    and as an aside, i hate flash, refuse to install it on my pc’s, because it’s the most obtrusive piece of software ever written. well, maybe it’s not the software, but the web developers, that think everything has to flash, play a movie and blare sound as soon as a website is visted, when all i wanted to do was to read an article.

    no thanks.

  26. Gabriel says:

    Thanks for the interesting blog entry that helps give some more ideas about how to use Process Explorer and Process Monitor for troubleshooting problems with software running on Windows.

    As soon as I started reading the problem the first thing that came to my mind was that it was a problem caused by Adobe Flash player.  I’m not surprise that you were not able to determine the root cause of that problem beyond tracing it to Adobe.

    However, I am curious about why the Windows UI was unresponsive simply because of high CPU utilization.  As some of the other readers have noted, the Windows scheduling algorithm should allow the user’s actions to receive enough CPU time, to do something like open the Start menu, even if there is another task running that is using all available CPU time.

    Usually when I see a system where Flash is using all the CPU time it will cause my browser (Firefox) to be unresponsive, but the Windows UI itself will continue to respond to user input.  I would think that in Vista using Windows Desktop Manager there should be even less reason for a misbehaving application to make the Windows UI itself unresponsive.  Also I’ve run various compute intensive programs that use 100% CPU time without causing the Windows UI to become unresponsive the way you described (30 seconds for the Start menu to open).

    I don’t know if you can comment on this at all or not, but I think it is a question that several people have posed and it would be interesting to know if you have any idea why the Windows UI was unresponsive in the problem you were troubleshooting.

  27. Ailurophobe says:

    Personally I disable video thumbnailers everywhere it is possible. They are commonly broken since they rely on being able to use third party code for things it wasn’t actually designed for. And many DirectShow filters don’t really support the full feature set of whatever they are supposed to handle to begin with turning thumbnailing into an automated crash generator. And apart from giving nice pictures for marketing, thumbnails are invariably totally useless. I mean really, what are the odds of a randomly picked image on a video file being anywhere near as informative as, a totally random example (not), the filename. Somewhere below 1% unless you have really bad naming conventions, I’d guess.

    As a Firefox user, I have the AdBlock extension installed for the express purpose of being able to disable broken Flash on web pages. Mark is probably stuck using IE but maybe there is something similar available. Not much that can be done about the web pages that refuse to work without broken Flash apart from avoiding them, though.

    Sorry if this ended up as a double post, I had a little technical problem.

  28. Rune says:

    Doc wrote:

    "Considering all that? I’m sorry Mark. Vista should be good enough to prevent this from occurring."

    _How_?

    Until you can answer that simple question, you are just a troll.

    Rune

  29. Rune says:

    Gabriel wrote:

    "As some of the other readers have noted, the Windows scheduling algorithm should allow the user’s actions to receive enough CPU time"

    How? By increasing the priority of explorer.exe? What if some plugin in explorer.exe runs away with the CPU? Instead of running at the same priority as other processes, you now have a high priority process eating CPU cycles…

    Possible Solution: Always run Task Manager (because TM is usually stable) at a higher priority, thus allowing the user to quickly shut down normal priority processes. (this still does nothing for the average Joe)

    Rune

  30. Keff says:

    Everyone blaming the scheduler: There’s no way modern scheduler could have hardcoded process names. If you want to give a process a priority – well, that’s what priority is for!

    Explorer.exe can’t run with elevated priority, because then OS would focus on running the shell more than on running your applications, which is what you buy PC for. If a process requests higher priority and then hogs CPU, it is application author’s responsibility, and he is the one to blame – call Adobe and Roxio, instead of blaming MS.

    On the other side, if the processes have same priorities, then a reasonable scheduler should give time to both of them even when other app goes berserk.

  31. Rune says:

    I have heard so many bad things about roxio’s product, and this for me is just another nail in the coffin. I just don’t trust them.

    Mark, look at the filter drivers they install for your CD/DVD devices. I think they might contain one or two surprises as well… (that you may or may not notice as time goes by)

  32. o.s. says:

    ‘A few weeks ago my wife complained that her Vista desktop was not responding to her typing or mouse clicks. Given the importance of the customer, I immediately sat down at the system to troubleshoot. ‘ Hilarious Mark just hilarious!

    I had a question though. So since ‘I had seen from the traces that the thumbnail cache generator had come across an AVI file that caused it to load the Sonic demultiplexer’ is true does this mean that the component went into an infinite loop trying to create a thumbnail for an avi file?

  33. Bob says:

    Interesting. Thx for sharing the troubleshooting process, I learned a lot. But I wouldn’t submit that for an "I’m a PC" video :-)

  34. fsr says:

    This blog is excellent to assess current real-world support: I doubt Microsoft Support would be able to troubleshoot that problem, and it would be impossible for a ‘poor’ OEM supporter, like Dell or HP, to troubleshoot it through popular email, chat, phone or VNC support; and if they were able to get machine on workbench, seems to me they would simply format HDD to factory settings and close ticket.

    Also, are you wondering why some costumers have VIP support and other’s don’t ? Get the answer from THE Windows expert !

  35. JF says:

    Thanks, Mark, very interesting…

  36. Guy Leech says:

    Great, thorough and amusing article,as ever, but what I find can work better with a CPU hogging process is to set its base priority to "Low" in Task Manager, or "Idle" in Process Explorer, so that its threads keep running but if any other thread becomes ready to run, it should get on a processor in preference to the Low priority threads of the wayward task since they will almost certainly be at a higher base priority (typically "Normal").

  37. Robert 'Groby' Blum says:

    I wish it was always that easy :)

    I’m still trying to figure out why my machine gets slower over time, without any obvious culprits in process explorer. (I.e. total CPU usage < 10%, plenty of phys. mem. available, and yet, sluggishness)

    So I’ll just go ahead and ask if you’ve got any ideas what to look for. (I mean, if *you* don’t know, who would? 😉

  38. Scott Johnson says:

    Mark, you state that "only see this kind of problem in special circumstances", but with Flash and 3rd party codecs becoming so prevalent, this is something that I see several times per month on different systems.  Both on XP and on Vista.  For servers I agree: it generally doesn’t happen all that often.  But for the less controlled environment of a personal workstation, it happens very often.

  39. Paul Renaud says:

    This is a great example of how Vista has failed to address the sheer and growing complexity of the Windows environment.

    Most of us geeks would have failed to get as far as you did, so what is the non-geek down the street to do?

    Rebooting and encountering the same problems over and over again is hardly a customer experience that will engender loyalty to the platform.

  40. Cristian says:

    I love your posts, man! Process Explorer is a wonderful tool!

  41. Gabriel says:

    @Mark

    Thanks for the additional explanation that makes it clear why you experienced the behavior you described.  Do you know why these threads were running with priority "Above Normal."  I tried running a webpage with Flash (youtube) on my Win XP system and the base priority for the IE process was 8.  Is there something different in Vista that raises the base priority for multimedia applications?  Also when I was checking this I noticed that Process Explorer had a base priority of 13. Is this something you added to help keep Process Explorer responsive when troubleshooting these kinds of problems?

  42. Johnny Sutherland says:

    Thanks for taking the time to explain this with the screenshots. Much appreciated. A great post.

  43. Pentti Hirvo says:

    So Vista was slow. No news there.

    Q: Doctor doctor, my Vista is slow.

    A: Don’t use it then.

    The take home point here would be that if you really insist suffering Vista, you’d better marry someone bright bloke like Russinovich to trouble-shoot it every other day.

  44. Wayne Robinson says:

    Really good and detailed investigation as always Mark.

    Thank you so much for sharing your experience with everyone.

    Cheers

  45. Lars Jeppesen says:

    @Mark

    >No, but it’s not uncommon for programmers to >program their threads with higher priorities >with the intention of making their operation >more responsive. In most cases programmers >should leave their threads at the default >priority.

    So this particully problem (dllhost.exe) was actually cause by two different bugs.

    1. The 3. party codec could hang in an endless loop.

    2. The priority was incorrectly "above normal", especially for a background thumbnail updater.

    But is bug 2 actully a problem in windows ?

  46. fsr says:

    Vista is great if you have the hardware, but troubleshooting it is as difficult as XP. I am just a Windows experienced user and I doubt I would be capable of using PE like Mark. If I would have found such a problem and if nothing was logged in event viewer, I would start excluding third-party entries with msconfig (or with msconfig in combination with autoruns), and once I have found the problematic add-in, I would have updated/downgraded the program or simply discarded it, replacing it for another.

  47. Angus says:

    ProcessMonitor really is a wonderful piece of software, thanks for it. I just used it yesterday to discover what was preventing me from installing Visual Studio .NET 2003 on my laptop. The laptop has had multiple install/deinstall cycles, and one of them had left incorrect product information in the Installer’s registry settings. ProcessMonitor made it easy to look at what setup.exe was doing just before it reported a failure to find vs_setup.msi, I found the offending registry information, moved it out the way, and the install worked correctly the next time.

  48. Doc O. Backbush says:

    Rune wrote:

    "Until you can answer that simple question, you are just a troll."

    Sure Rune, take the easy way out label me a complainer. Just someone who knows a little bit more than the average user, but not in a position to do something about it.

    I don’t work at Microsoft. I don’t write OSes for a living. Maybe what I proposed is absolutely impossible to do. But that still doesn’t change the fact that the end user is not going to be married to a Debugging Genius if they insist on using Vista.

    Check out my blog. I say the things that need to be said, especially if too few are saying them. Vista should NOT behave like this. That needs to be said. What will be done about it? Who knows.

    But the first step to dealing with a problem is acknowledging it. And Rune? Putting your head in the sand with such demeaning terms like "troll" doesn’t solve the problem, it only exasperates it.

  49. Scott says:

    Mark,

    Thanks as always for an informative how-to…

    I poked around on my system (XP Home SP3) and I know I have the latest Flash, and it’s Flash09f.ocx from March 2008 (yours appears to be Flash09d.ocx from the screen caps).

    I suspect yours is NOT the latest version.

    On XP, it’s stored in C:WINDOWSsystem32MacromedFlash and I’ve had problems with newer versions leaving older ones present in that directory and I’ve had to do manual cleanup several times.

  50. Srikanth says:

    Mark, this is a fantastic narration of a great troubleshooting experience. Invaluable for those reading this.

    And I understood more about ProcessExplorer, you’re awesome :)

  51. Jason says:

    "That’s because the computer was a dual-core system and so to consume 100% of the available CPU cycles a process would have to have two threads, each hogging one of the cores."

    Of course, with browsers moving toward creating individual processes per tab/plugin, a couple of tabs with malformed flash could easily consume 100% CPU, even on a multi-core system.

  52. Monk E. Boy says:

    Just to follow up on what Scott said, Adobe likes to issue silent updates to Flash. I would recommend uninstalling Flash, downloading the latest version, then installing Flash. In all likelihood you will end up with a newer version that was installed. Whether the new version will solve the problem or not is anybody’s guess – the wife probably just visited a website with a malicious Flash widget.

  53. Robert Lacroix says:

    Interesting article, but non-IE browsers seem to have problems with the SSL certificate of blogs.technet.com an so some of the pictures don’t show up.

  54. Dimitris says:

    Wonderful work Mark, thanks for sharing this info with us.

    I wonder why people keep saying Vista is slow and it is Vista’s fault and so on.

    1st, that is not at all constructive, hardly even a complaint. especially when, as mark said, nothing here is vista specific.

    2nd, OS’s have different uses, and Vista is very good for what it offers. Mac is completely different, Linux as well. (I currently have Vista, XP, MacOS x, open suse, and ubuntu.)

    3rd, Runaway processes happen in every system, and taking into account that windows has by far the most 3rd party programs availiable, because of its popularity, i think it handles that really well. I have seen my mac hang cause of flash.

    4th, having a better scheduler does not sound easy. its one of the things that if you make too complicated, would make everything slower.

  55. Dr. House says:

    Beyond trolls, super-geeks, and, non-geeks,all of you people read like the goober picking, sorry…eating, no life, no dates, and certainly, no clue buggers whom beg to be beaten simply for being. Members of those E-C High School clubs lasting long enough for the real people to leave, so the janitors didn’t have to retrieve your high-water pants or long concealing coat from the flagpole? Now that I have all of you angry, ask yourselves this; "at whom, and, why, am I mad?" The answer, of course is moot because I don’t care. I see brilliance, all of you are intelligent people however, what good becomes instruction when  real-people who really need your help, can’t find these smart pages. Mark? Are you there, Mark? Oh’ let us not punish Mark,let us pretend the English language hasn’t become castrated to electronic acronyms which, have very little meaning for the end user."Exasperates," the word is "exacerbates" my computer nerdy pals yet, it’s Mark’s Blog. Exacerbating the problems of the scared end-user, hacking registries, dll’s,I-frames, HDD’s, adding Marks’ very nice detailed diagrams, deary me? Whatever would I do without something to complain about? Hmm’ to be, or not to be heard and understood by those whom would…still, run your pants up the flagpole, but say thank you, afterwards. Hughe  

  56. DC says:

    Always a pleasure to read your articles Mark, you make my job easier

  57. Gis Bun says:

    Isn’t Mark’s wife lucky that she has an expert to help her out. :-)

  58. Bobw111 says:

    Amazing.  30+ years in computers and I am still reading articles on how to trouble shoot run away processes.

    Maybe the ancient round robin technique of so many ticks per process wasn’t such a bad thing after all.

    Mark, Thanks for a very informative article.

  59. Richard A. Steele says:

    Re Flash hogging CPU:

    Please vote for the bug at
    https://bugs.adobe.com/jira/browse/FP-417.  Though  I’ve seen it specifically with Firefox, I’m not surprised it also shows up when using IE.  I, too, started using Flashblock because of this very defect.

  60. David says:

    I still hate how Vista updates the thumbnail cache even when I’m not browsing around in Windows Explorer. Comes out to a couple consecutive minutes of disk activity each day that I can’t easily disable.

  61. MH says:

    Mark, did you read through this post properly before publishing? You surely made an obvious mistake here? 😉

    ‘At this point the only options I had were to uninstall Flash and leave my wife with a degraded web experience, or terminate IE to stop the current CPU usage and hope that it wouldn’t happen again.’

    When you wrote ‘degraded’ you surely meant ‘improved’?

    Seriously though, nice work as usual. Just one request – more frequent posts please!

    MH

  62. CS says:

    After all these years of evolution, computers, operating systems, and software overall, are all still merciless with the average user, the ones who should be using computers, not being afraid of doing it…

    What if Mark’s wife wasn’t married to him? Would she have been lucky and stumbled over an excellent post like this one?

    It’s not only the PC… not only Windows. It’s the concept: does the industry really expect average users that are able to deal with process managers, threads, and the like?

    But wait: things get worse: I am pretty sure that the average support staff does not even have half of Mark’s knowledge

    Let’s face it: we’re still light-years away from people smoothly and naturally using computing devices for their daily chores… at work, at home, and on the street.

  63. Patrick J. Burwell says:

    We DISABLE flash, and any attempt to install/update flash, on our Terminal Servers for this VERY reason. Flash component programming is too easy to foul your system and we certainly didn’t want our client performance ratings to be affected because of an advertisement.

    Patrick J. Burwell

    VP, Technology

    iQor, Inc.

  64. Stephen Walker says:

    Video thumbnailers, including the one built in to XP, tend to have great problems when thumbnailing an AVI file with a broken or missing Index Block at the end of the file. This could be the cause of the second problem.

  65. John says:

    <blockquote>

    A few weeks ago my wife complained that her Vista desktop was not responding to her typing or mouse clicks. Given the importance of the customer, I immediately sat down at the system to troubleshoot.  It wasn’t completely hung, but extremely sluggish. For example, the mouse moved and when I clicked on the start button the start menu opened after about 30 seconds.

    </blockquote>

    I thought that was the standard Windows behavior!

  66. Ryan says:

    Perhaps XP with a Nero install would reduce your time spent troubleshooting.

    Jokes aside — is your wife using the Roxio functionality?

    Or does she just burn CDs/DVDs like most of us?

    Do you really need 640mb of bloat for unused functionality?

    Is there a well developed application with a smaller footprint which performs whatever functions she needs?

    Mark — I presume from your excellent applications that you understand such an argument. I beg of you to use your influence to echo such a concept throughout Microsoft.

  67. Adrian Wilkins says:

    I find that I have to turn off thumbnailing on Vista completely, or Explorer just isn’t stable ; many has been the time where just viewing a folder, or even the parent of a folder, containing media files, makes Explorer drop dead.

    My take on it is that you shouldn’t be allowing a task like thumbnailing to be killing a whole process, especially one as critical to your productivity as Explorer ; why isn’t the process insulated from it?

  68. blagoev says:

    There is an easy way to disable shell addons. Check this tool ShellExView (from NirSoft)

  69. TG says:

    Very interesting article.

    But I wanted to troubleshoot some slowness on a company laptop and it seems the new 2.0 version of Process Explorer doesn’t work on Windows 2000.

    I get the following error:

    The procedure entry point getnameinfo could not be located in the dynamic link library WS2_32.dll

  70. GagMe says:

    Good troubleshooting efforts, Mark.

    But Windoze-Live-searching?  I sincerely hope you were attempting humor.

  71. Jeff W says:

    Very good sample of how to use Process Explorer. I consider myself a pretty solid troubleshooter and I learned a few things here.

    I’ll follow your example except my searches will probbaly be run on a site named for a very large number.

  72. Jeff W says:

    Very good sample of how to use Process Explorer. I consider myself a pretty solid troubleshooter and I learned a few things here.

    I’ll follow your example except my searches will probbaly be run on a site named for a very large number.

    FYI – I have seen these sorts of issues with Flash and 3rd Party Codex forever.  I usually assume it’s sloppy Flash code but it would be nice if  Flash trapped it better.

  73. Yuhong Bao says:

    On one Duron 850/Win2000 system, I often had to reduce the priority of Firefox after the system became unresponsive for seconds due to it consuming 100% CPU time.

    Anyway, thanks for all of "The Case of…" articles. Too many people blame every problem they had when running Windows on Windows itself, while more often the problems were really caused by 3rd party software.

  74. Martin says:

    Could be a good idea to add a Refresh button to that "Stack for thread xxxxx" dialog box to eliminate the need to close and re-open it. Thanks for great blogging!

  75. Adam says:

    Awesome to see a come back of The Case of… Thanks Mark!

  76. Lorraine says:

    Oh woe is me!!! Why couldn’t I have come across your most excellent explanation BEFORE I spent two days attempting to fix a friend’s (first mistake) slooooow and Trojanized machine.  :-(

    I refuse to even look at a machine unless I can install Process Explorer but realize that until now, I have only touched the tip of the iceberg in what it can do and what it could have shown me. This article gets printed on drilled paper.

    Thanks from long time fan.

  77. Shahbaz says:

    Great article Mark. It’s got to be simpler than this & handles by the OS itself as several million of us all have similar customers running Vista OS :) .

    Fixed in the next Desktop OS release ?

  78. Intentsly says:

    I hope you forwarded your findings to Adobe and/or Roxio.  That may actually be enough grease to have either come up with a solution.  Perhaps even provide solutions to support problems they’ve had go unanswered for a long time. Good Job Mark!

  79. Jeroen says:

    suspending a process helped me today to get a system free from malware, saved a reinstall. remembered that from this post, after the normal struggle with an agressive malware got me almost stuck. thanks again.

  80. David De Roxio says:

    Hello,

    I am a Roxio support agent.

    I see that this issue was resolved. However, if you ever want to optimize your computer’s performance, this article should help adjust Creator:

    http://kb.roxio.com/content/kb/Creator/000103CR?set-locale=en

    Regards,

    David

    Roxio Support

  81. SRS says:

    From Roxio support: "Locate RoxWatchTray10 (or whichever version you are using) under the Startup Item column and uncheck it. " – i.e. "To make your computer run faster, don’t run our software". Roxio should be commended for honesty.

  82. Vinh says:

    I’m very like the processexplorer and process monitor. I’m learning how to use them professionlly like Mark by check out his blog every month, but everytime he about process monitor, I’m getting confuse. Well, how can I master using process monitor? or I appreciate any suggestion.

  83. Miksa says:

    Obvious wrote:

    > Perhaps XP with a Nero install would reduce

    > your time spent troubleshooting.

    Nero is just as bad. It has installed it’s own MPEG4 demuxers and codecs since at least version 6 and this has caused no end of problems playing videos. I usually reduced the position of the Nero codecs on directshow filterchain using Zoomplayer’s filter management, so Dshow would use other codecs that I had installed, like DivX/XviD or ffdshow.

  84. Chris Hills says:

    Might I suggest for altering DirectShow filters, a program called "DirectShow Filter Manager" (I wonder why sysinternals didn’t come up with this!) available from http://www.free-codecs.com/download/DirectShow_Filter_Manager.htm. It makes it trivial to register, unregister, and alter the merit (a DS term – with values ranging from "preferred" down to "do not use") of each filter.

  85. Mat says:

    I agree with Rune.  Troll.

    Anyway, thanks Mark for an inspiring, informative post.

    It never ceases to amaze me how resistant people are to new operating systems that are supposedly ‘forced’ upon them!

    Mark has clearly and concisely illustrated how it’s purely an application issue, and how the operating system is built by design not to behave differently for applications that demand the CPU’s attention.

    Whilst it may affect user experience, we must remember that it’s the user who installed Adobe Flash. They clicked the little button that says "I accept".

    Well – maybe not the user, maybe the administrator – in which case it is him or her who should take responsibility.

    Either way, it’s not the duty of an OS to make decisions about what gets priority. I use my computer for some vast graphics and audio number crunching sometimes, I would get pretty pissy if Vista didn’t respect my needs as a user of that computer to do the work as quickly as possible!

  86. Marc says:

    The comments about efficiency, scheduling, and round robin algorithms are interesting. Having worked as a systems programmer, I agree that it’s not easy to code these things so they’ll work for everyone without unintended side effects. However, the algorithm chosen has to be appropriate for the user base and the problem set. QNX (a real-time O.S.) was completely different from the various Unixes I used for this very reason.

    What I’m suggesting here is that the scheduler for Windows desktop users needs to be changed to reduce the damage runaway apps can cause, rather than trying to produce the highest possible performance. Requirements can conflict, and as Steven Wright says, "You can’t have everything. Where would you put it?"

    Sometimes you have to make compromises. With raw iron cheaper and faster, maybe it’s time to change our mindset about performance and think more about usability under a variety of circumstances, including runaway apps.

  87. Warren says:

    My Sloooow Vista System is at startup right after I log in. I can’t do anything for a good minute and a half. Is there any way to get Process explorer to log what is happening at this time?

  88. John H says:

    Interesting insight from everyone which helps explain what is "hosing" the CPU. I am running XP, IE6, Roxio 6. Knowing I need a new dual core laptop with 4 meg memory, probably this Christmas, I don’t like the reactions from Vista users and this will be my only OS choice offerred. I RUN msconfig and uncheck alot of processes and this seems to help, but not all are listed, so thanks Mark for P.E.

    I constantly clean the registry with RegScrub XP, scan with Lavasoft AdAware 4 times a day, clean my TEMP folders, and run Spybot Search/Destroy. I am sure we all get sick of doctoring up the health of our PC’s. It is still a priviledge to have all this iNFO at our fingertips EVEN IF IT TAKES TIME TO GET IT!

  89. Kevin P. Rice says:

    "my wife hadn’t noticed it because the remaining CPU capacity was enough to mask it (another good reason to buy multi-core processors!)"

    — I have a program with this statement because if 50% of my processing power (or 25% on a quad-core) is unknowingly being stolen… this could go on for weeks without being noticed. I didn’t pay extra $$$ for a quad-core just to cede one or two cores to rogue threads!

    Perhaps a monitor is needed that warns when a long-running thread is hogging CPU cycles? Of course, one more info balloon in Vista would just about send me off the deep end!

    Ideas?

  90. David De Roxio says:

    Hello,

    As an amendment to my post above:

    Roxio wants users to get the most out of their software. There are a few features and configurations that may not be suited for every machine. That’s why Roxio supplies articles to help adjust and optimize your system.

    Imagine a car that didn’t have an adjustable seat or mirrors. Software, and the environments they run on, need to be just as flexible.

    Regards

    David

    Roxio Support

  91. Chris Kreukniet says:

    Thanks for sharing your problem and solution. It’s nice to read how you troubleshoot your event.

  92. SRS says:

    David De Roxio: Nice sentiments, but you did say earlier "if you ever want to optimize your computer’s performance…" and the link you supplied suggesting "optimiz(ing) your computer’s performance" by disabling bits of Roxio’s software – so Roxio believes and documents that computers run faster without Roxio software installed.

    Your analogy of adjustable car seats and mirrors doesn’t really apply, as adjustment of these components (or even the removal to stretch the point) doesn’t change the speed of the car – but the safety of the journey.

  93. Richard says:

    From the screen-shot, it looks like Flash had just been updated, and was probably trying to display a notification message. "FlashUtil9d.exe" is the Flash ActiveX updater utility in C:WindowsSystem32MacromedFlash.

  94. jcg says:

    John H – you are not stuck with Vista on your new PC this Christmas. most vendors should be able to preinstall a XP "downgrade" with your chosen Vista OS, particularly if you visit their web site as a "small business user" (very small = one employee). i just went to the Dell web site, and they will do this for you, even for home users (on some models).

  95. jcg says:

    scheduling algorithms don’t seem that hard to figure out. there is already a setting in windows that lets you give priority to foreground processes (vs background). it seems that windows may not be interpreting foreground the same way that users do. from a users perspective, the priority of any OS would probably be: 1. mouse and keyboard (if you can’t override what’s going on or start something new, you’re staring at a vista-colored brick), 2. things that have a non-minimized window, 3. things that have a minimized (taskbar) window, 4. things in the tray, 5. stuff i can’t see running (background).

  96. SPowell says:

    Wow, my wife would never let me delve into her PC that deeply to find a bug, she would have restarted it herself before I would ever have been able to get to it, she’s a bit more impatient than your wife apparently. I have to say, these blogs of yours are certainly helpful in creating new troubleshooting steps for similar issues we see as Network Engineers when we’re out at a clients location. Your tools are immensely useful and have helped us in more situations than we can count. Keep up the good work, and keep that sense of humor going.

  97. Victor C says:

    This was very helpful. I am always being educated as well as entertained.

  98. Anonymous says:

    Good article, but I bet you actually googled the CLSID :)

  99. gki610 says:

    Hey, folks, Flash has a new one: 10a.ocx. I had software issues since a tech ruined a memory stick, and I found it, removed it, and now nothing crashes. I thought Flash was going to kill me.

  100. P says:

    I’ve noticed IE spinning when I load lots of tabs (typically from browsing through RSS feeds of blogs and clicking links for later review).

    My analysis is that so many sites feature Flash animated ads that there just aren’t enough cycles to service them all. As I close tabs the usage drops down again.

    ProcessExplorer is a great tool for tracking these things. I also used it looking at slow startup on an old XP Tablet to determine that Windows Update was thrashing reevaluating hundreds of old $NT… artifacts over several years of system updates.

    Thanks Mark!

  101. Dave Minnick says:

    A lot of information that needs to be carefully digested before attempting.

    How about a printable version without the comments, some of which are well intentioned but not necessary. After building a new system I was about to yell at Intel because the system would hang up if I tried to use it before it got done running a 100 or so items. As I have both Flash and Roxio installed I need to look deeper as to the primary cause. I installed Process Explorer and it is not yet telling me what is using up the dual processor.

  102. Sam says:

    I have 2 things I want to share too.

    1) Thumbnail generation is working for windows explorer. So if that demultiplexer is hogging CPU, you can’t just boost the user input by putting the explorer.exe to higher priority because you’ve just boosted that high consumption process even higher.

    It is already much better in Vista SP1 than pre-Vista SP1, which if you have an error in your video file the explorer most often would just hang and get you a COM Surrogate error.

    2) people just have a misconception of boosting user interaction process would solve everything. System resource is a scarced thing. The OS is under the presumption that if an application is running on "above normal" priority, it must have a good reason for it. It’s a reasonable presumption because, say, if you’re burning a DVD, you’re most like don’t want to disrupt the burning process even the user interaction is a little bit sluggish at the time. FYI, when I burn disc on my Mac OSX, guess what? The system slows down too.

    Multicores CPU could helps sometimes, but most programs nowadays are multi-threaded so it wouldn’t help by much. The burden of judgement for which process should run higher than the other shouldn’t lay on the OS. The user and the application writer should take responsible for that matter… though I have yet to see a solution that would let user manage process priorty that would be easily understand by average user. (i.e. my grandma)

    Adobe, out of the good intention, probably wants their Flash player plays as smooth as possible, thus the "above normal" priority, but when it failed it failed very, very hard.

  103. Q says:

    I have recently gone through a very similar debugging process using Process Explorer, for my partner, and found that the culprit for a slow unusable machine which has crashed many times and lost hours of work is again buggy Roxio software: the "media watch tray".

    Looking this up, I found many sites saying never to uninstall Roxio software, as it uninstalls some components which are written by Roxio, but part of the Microsoft OS and used for CD burning.

    I think it is astonishing that someone from Roxio support can claim here that the problem "is resolved". There must be thousands of people who have installed this software and haven’t got a hope of finding why their machine is now broken. My partner teaches IT, but using Process Explorer is a bit advanced even for a software egineer like me.

    I’m also astonished that they can suggest hacking the registry using msconfig to disable their software, instead of providing a tool that removes it safely.

    No software should install components that aren’t necessary for the basic functioning of the product without giving the user the option to opt out at install time.

  104. Maureen Sampson says:

    It has taken me forever to read all the comments relating to this matter and to be honest it is all too technical for me to understand (I am a silver surfer).  I could throw my computer through the window because of its slowness and also the freezing of the screen and the only way I can close it down is manually which I understand is not a good thing for the computer system.  I get so frustrated and my blood pressure soars!!!  I try to get help and information through the internet but in the majority of cases my computer freezes before I have gone anywhere.  The problem seems to start after I have logged on.  The arrow moves but nothing happens.  Even in the middle of typing a works document the printing stops.  I do have Roxio installed on my machine (I think) because a Roxio pop-up comes on when I start my machine.  Is there anyone I could speak to or email who could help me with this problem please.I am afraid my usual advisor is at uni until beginning of December.  Someone please help me.

  105. Patrick Julien says:

    Hi Mark,

    What would you recommend to track down this problem:

    I have many vista machines and on one machine, just one, where the transition between sessions take several seconds.  This happens on login, logout, ctrl+alt+del, UAC prompt or any other session switch, for example, the infor card manager.

    The reason I’m stumped with this problem is that the session switch is a very akward spot to get a problem.  The session is in the process of being switched so it’s hard to actually monitor the system in that state.

  106. JV says:

    I love process explorer and process monitor; I used to use other shareware apps which did the same thing before the free sysinternals tools came along.

    I’m more minimalist than most users; but I find uninstalling Macromedia Flash and seek-and-destroying any auto-updating or daily version-checking utility (Roxio, I’m talking about YOU) goes a LONG WAY to making my systems more stable, responsive, and overall leaner and meaner and more pleasant to use.

    I’m here to seek out the content the google search results told me I was interested in, I have no time for your stupid animated advertising that is just AN OBSTACLE in my way.

    So like one would run "top" continuously on a linux computer; I also run process explorer continuously in WinXP to see "what’s stealing my cpu cycles?!?"

  107. Erik Itland says:

    Somebody mentioned ad block for IE. I think it’s included in ie7pro, but I don’t know if it’s good, since I’m using Linux most of the time. Think it has features like getures etc also.

    Mark will maybe just make something smarter than that for what i know, it’s just in case anyone else reading this wonders :)

  108. a.k.a. says:

    Here’s another problem: What is it that’s preventing a reader from saving this blog post in IE7 through anything but the Print-as-XPS file option? I’ve noticed that with the NYT blogs as well, but here there’s no Flash ad that’s crashing the Save function.

    This is a big usability annoyance. Mark – You have the pull with MS to get their CMS programmers to work this out, right?

    Thanks!

  109. kman says:

    Go to add or remove programs and delete Roxio and Flash.  Or, get rid of Vista and go back to XP or better yet buy a Mac.  Case solved.

  110. eep says:

    I actually read the whole article because this kind of thing bugs me also.  The Flash issue would appear to be specific to the content of a particular page.  On one popular news blogging sites I used to leave the IE tab loaded all day and the auto-refresh feature of the web page would eventually cause lots of memory and CPU to be consumed after a day of refreshes.  The site frequently had active Flash content that was used for displaying advertisements.  So, perhaps usage patterns have something to do with the Flash CPU phenomenon, particularly if there is JavaScript or some such other scripting language that is attempting to control the embedded object.  

    On a lighter note, your quips about Windows Live Search made me think of a nickname that could supplant "google" as a search verb or gesture: wiggle.  Thrive Ventures has wiggle.com registered: http://www.thrive.com/

    Sorry, but I rarely wiggle, unless I think there’s a chance that some Microsoft-specific information might be more readily available there.  I use an iMac now and run Windows in VMWare Fusion 2.0, if I ever need to do anything in Windows.  I could have purchased a touch screen HP that runs Windows Vista, but I have an iPhone and I decided to immerse myself in the alternative geek lifestyle.  I’m loving it so far.

    Now, if only I’d bother to turn on my XBox 360…

  111. technogeist2k6 says:

    I had a similar problem with Roxio on my laptop. Flash is a recurring nightmare. If it isn’t bugs in Flash Player to begin with, it’s some badly coded flash content. (deliberate with some sites)

    Mark seeing as you’re now working at MSFT, couldn’t you push these issues up the food chain?

    We’ve been suffering this type of issue for many years, and still your employer insists ‘It’s the best Windows yet!’ Really?

    I’m a big fan of virtual server which helps to limit CPU resources, and so prevents the guest OS from achieving 100% utilization when things go awry. Perhaps you should virtualize your wife’s PC.

  112. MrPete says:

    Mark, there’s another situation where the scheduling algorithm makes systems fall apart.

    Any time an interaction throws off huge numbers of new processes, the scheduler becomes "priority fragile"…

    Right now I’m tracking down a similar system-killer. In this case, a script is cleaning up some unused ACL’s by using subinacl. That generates a ton of new processes.

    But the real problem is an interaction with Logitech’s LVprcsrv service. It hit 1.3 million (!!!) handles before I got the distress cry from my wife (family tech support: the ultimate NMI :) ) and figured out what was going on.

    Not surprisingly, Vista has pretty crappy performance when a process has eaten a million handles. But it sure would be nice if the UI had higher priority than scripts, etc.

  113. Danno says:

    I remember back in the day when I was programming in a mini-mainframe environment (yes, I’m dating myself ;-).  Back then, the OS’s scheduler (Primos) was very robust.  There were 3 primary components that were admin-controllable: Priority, timeslice, and keyboard burst.

    It seems that the NT4 scheduler model (still being used in Vista, according to Mark) is quite obsolete and needs to be redesigned from scratch.  While moving priorities up and down can help with sluggish response, it does address timeslice.  A process that is assigned a reasonable default, configurable timeslice by the scheduler won’t have nearly the impact on the system, since when it’s timeslice is up, another process can at least get a shot at the CPU.  Also, allowing for mouse/keyboard burst time will allow a foreground, interactive process (typing, moving the mouse, dragging/dropping) to get immediate response.  This should be default scheduler behavior, unless the computer has some unusual usage requirement (server, real-time data acquisition, etc.).

    IMO, there’s difficult to imagine a legitimate reason for there to ever have a 30-sec delay for mouse, keyboard response, unless a process/thread has been assigned real-time priority or an unlimited timeslice.  I wonder if any of the ‘nix OS schedulers have these features built in.  Perhaps someone more familiar with ‘nix internals can fill me in on this.

  114. Tim Reeves says:

    I’m sure you’re not allowed to say so, but another option might be use another browser besides IE!

  115. Kropotkins_anarchy says:

    >>>

    I’m sure you’re not allowed to say so, but another option might be use another browser besides IE!

    >>>.

    This is exactly that I did. Firefox + addons solved MANY of my problems.

  116. Jeff says:

    I have observed the same behavior when using the Live Writer application. The dual CPU usage goes to 100% and typing response slows to a crawl. After 15-20 seconds the CPU load goes back to normal and all is good for a few more minutes before this cycle repeats.

  117. ken says:

    how odd. I did a technet search for

    http://technet.microsoft.com/en-us/default.aspx?query=badbeef

    badbeef.

    nothing about bad beef. But the reason I’m looking for this is a SLOW system. And in the process of attempting to restore services to default settings.

    maybe I’ll find out what badbeef means.

  118. Tim Donovan says:

    Mark great post.  I too was having some issues with the wife complaining of slowness and your post made me think of checking for this and found the same problem with flash in IEFRAME.  I had an old version and upgraded. I didn’t have the other problem your wife did but upgrading flash has seems to fix my problem.  If I hadn’t read this I would have continued to blame Vista for this (and I’m no vista alarmist).  Obviously Microsoft can’t check for every 3rd party but flash is fairly ubiquitous,  I would think that there would be some sort of Windows update sort of alert to advise of things like this.  But I suppose legal issues intervene.  I make my living as a c# developer and have been on software projects long enough to see that there are trade offs for a successful program that lasts over years (ie. supporting backward compatibility and dealing with hindsight 20/20 complaints) Anyway now when I see similar issues with Vista I will use this technique and act accordingly but it’s stuff like this that makes a power user consider Mac and linux just to see what we’re supposedly missing.  I think they will find it’s lonely at the top too if they get there.  

    Anyways you’re tools have saved my bacon many a day and I salute you sir and am glad you are a part of the solution!

    Cheers!

    Tim

  119. ochiru says:

    Hi Mark,

    is it possible to trace and undo registry/file changes with Process Monitor or other Sysinternals tools when installing/removing software?.

    If it isn’t possible yet, I think that it would be easy to implement to your Process Explorer/Monitor. I also think that it would be "A Killer Feature" 😀

    btw. what’s with Adobe installing flash to %windir%SysWOW64Macromed? Who would want to install a browser plugin to a system folder?!

  120. Nelson says:

    Danno hit the nail on the head.  I was going to submit a similar post but he pretty much covered the multi-thread OS concept.   No "robust" OS should let ANY process run away with the CPU the way, at least, Microsoft OSs have permitted and apparently still do (due to a faulty and obsolete scheduler implementation) so that no thread gets starved of the CPU for 30 seconds.  If a thread hasn’t finished its work within its timeslice, it must wait its turn for another timeslice to resume work, barring some reasonable priority scheme (not 1/2 minute).  

    With the speed of modern CPUs, the user would typically not notice the CPU hog since they would get a timely response from the OS.  These concepts have been around for years (before Windows & even DOS) so Windows has no excuse for not having better task management.  

    I hate it when I click on something like the Start Menu and nothing happens.  So you click it again, and wait…and wait, and then, the Start Menu pops-up and then disappears as it executes both clicks, although I’ve never had to wait 30 seconds.  And yes, I have even learned to click it a third time so that the menu will be visible when the OS finally gets around to me.  

    @Dr. House, Hughe, maybe Denise Miller, or more likely J@CKA$$:

    Before ripping into somebody for making a mistake, make sure you know what you are talking about.  Exasperates and exacerbates have practically the same definitions and are synonyms, so Yahoo! (I had to throw them in there for fairness/balance) synonyms and apologize.  

    Also, don’t lie and say you don’t care.  You cared enough to post didn’t you?

    BTW, (<–acronym around before electronics so I’ll assume it somehow isn’t considered English language castration by you) there is no such thing as "high-water pants" anymore.  The kids now call these shorts:)

  121. Walter says:

    Nelson comments:

    "Exasperates and exacerbates have practically the same definitions and are synonyms"

    This is exactly the sort of exasperating mis-"information" that exacerbates the growing problem with careless and imprecise use of language. :)

    Exacerbate: to make a situation worse; more violent, bitter, or severe

    Exasperate: to annoy or provoke anger

    Agree with you, though, about "Hughe". His diatribe is now immortalized in my collection.

    …Great article, Mark.

  122. RU says:

    I’ve been a big fan of Mark’s since I discovered Winterals years ago. I love the logical organization of his apps in areas which MS has always needed major help with. (though I’ve never been able to get your access group dumping app to work in my domains <cringe in frustration>).

    After reading a number of these comments after Mark’s story I think we have to say that Windows’s memory management needs help. Not only should we be able to run a memory gatekeeper app which has a set of rules that looks for issues like runaway CPU hogging processes which has self identification (loops of 1000’s of cycles that never end), you can drill through various categories like the stack flow, dll cpu process levels, etc, when they’re happening. But also, a way to identify memory allocation to components to identify excess, a 100% detailed and accurate breakdown of how much memory Windows is allocating at start and after apps are run, so you can see why windows needs 800 MB on a fresh startup, but after an hour why is it at 1400 MB, or whatever the issue.

    I wish there were companies that would produce memory management subsystems which we could swap in as a superset to correct these problems or help granularly diagnose them.

    As far as I’m concerned Mark has done a great job at unveiling some of the mystery with Windows subsystems with his apps. Maybe someone at MS will get smart and make him a project manager of some of these subsystems so the next version of Windows can finally improve it’s efficiency rather than just cosmetically.

  123. Hieronymous Cheese says:

    My computer has been horribly slow for the past few days. The mouse was sluggish and, as noted above, pressing buttons resulted in an extended delay before anything happened. The keyboard kept missing letters when I typed.

    I usually keep the Task Manager open, so I can keep an eye on things, and am therefore familiar with the processes which "should" be running under normal circumstances. Over the past few days I noticed the appearance of the aforementioned "dllhost", which I don’t ever remember seeing previously.

    Noticing the high CPU usage of Firefox, I decided to change the priority to Below Normal. However, I was surprised to find the priority already set to High. I changed it to Low. Checked again, and it was back at High. Tried several more times, with the same result. WTF??? Nothing like this happened before.

    So what changed a few days ago?

    I installed Silverlight.

    Could this be the answer? Bet yer house it could!

    After uninstalling Silverlight, everything is back to its normal speed. Firefox no longer insists on running at Above Normal priority, the mouse zooms around the screen as it should, and the keyboard is functioning correctly.

    What the hell are MS playing at with this nonsense?

  124. Sanjay Melinamani says:

    This is really a great info for professionals. Applications like Flash, Roxio, AOL etc are the culprits for Windows bad behavior. Unfortunately that is not what others see it. So, I wish Windows team would properly handle such scenario and a threshold alert say 70% CPU. If it goes beyond 70%, let Windows report the info you collected using all your knowledge and the tools to the end user. This helps a lot.

  125. Sankaran says:

    Even in linux (Ubuntu, Fedora …) I am seeing flash taking 100% cpu time.

    Why blame windows for it?

  126. Ilya Barskiy says:

    Hey, Mark,

    Hope you’re still monitoring this.  This is kind of an appropriate place to post it.  There is some kind of a bug or "feature" (clearly not working right) that makes CPU usage on my laptop (and other people’s machines) to spike heavily thanks to explorer.exe (windows explorer process) when there is My Web Sites on MSN shortcut present in the system.  I don’t know where it’s from, and once moved to recycle bin the usage of explorer.exe goes from 50% on my new Vista x64 installation to something like 1-3%.

    Call stack shows all MS dlls

  127. GH says:

    I am sorry for english.

    after all years, A giant like microsoft is developing an OS which is Windows and it is getting slower or crashing because of stupid web pages those high loaded with flash files, or the registry is getting corrupted because of bad un/installations,etc.

    What the hack thousands of Microsoft OS developer are doing ? are they know thinking ? or designing ? or microsoft management is sleeping ?

    It is shame , because that BIG VISTA is getting responceless because of stupid Flash files in the web pages ?

    Shame.

    I am not anti windowser or lover of mac or lover of linux.

    I just want please, drink a black coffee and think and design a working OS, not the Kinder Garden OS.

  128. Flatliner DOA says:

    Interesting, I’m now running the Windows 7 Beta and i’m experiencing a similar perf hit in explorer when viewing folders with AVI’s in them. Seems like there is a rogue filter, which seems like it is also affecting Media Player and Media Centres performance enumerating files.

  129. PuddinPants says:

    If you are running Flash 10 and still experiencing the problem, then drop back to Flash player 9.0.151.0.

  130. skeptikal says:

    Regarding the ‘Sonic’ application in the article, this is not actually part of the Roxio software package.

    I believe Roxio bought them out or something, I dunno exactly the deal there.

    You got the Sonic software most likely as part of the software with the computer, I’m guessing you had a HP or Compaq.

    If you look for updates to Sonic it just kicks you to the Roxio page, my assumption is that Sonic is legacy product & they’d prefer you just remove Sonic & install Roxio.

    You might be able to get an update by going to HP/Compaq and doing a software update instead of trying to do the app by itself.

  131. Sam says:

    Surely the correct way to fix IE is to install Firefox or its variants, and flashblock?  Then you can ONLY allow flash you WANT to run, saving your DL as well.

  132. Ben U says:

    Wow. Slow Vista System? Let’s go through twenty steps no normal human could ever accomplish, then decide that simply solving half of the issue is good enough because you’ve got enough raw hardware that the user won’t notice.

    Awesome.

    Why not just get a quad core? Then you can go another two years without fixing a Windows issue.

  133. mojo says:

    i run flash in firefox on linux and periodically have seen similar problems stalling the browser.  in a fit of youtube click-itis i may see several tabs with flash.  this has not been so much a problem since getting the 64bit native flash plugin that they finally released.  previously i had been using nspluginwrapper to run the 32bit version.  the usage still maxes some times and the browser becomes unresponisve, but the other issues i had are gone.  with the 32bit plug-in, some times after a while the tabs will not load the flash without a few reloads, and other times all the flash content dies leaving grey boxes in their place on every tab that had some.

  134. mojo says:

    sure wish flash were not closed source 😉

  135. Fran Taylor says:

    Anyone who knows me will know that I’m the last person to defend Windows, but these sorts of problems happen with EVERY operating system.  

    For example, on Unix/Linux, an application can pass junk to the X server and totally hose the server.  You can move the mouse but otherwise it’s just dead.  You can kill the X server or restart the system, both are bad.  I don’t know OSX well enough to develop a scenario, but haven’t we all seen more than our share of that spinning beach ball.

    Schedulers are black voodoo magic on every operating system.  Even the most minor of ‘tweaks’ can have grave consequences.  Look at the developer’s mailing list archive for any open source operating system, and you will see LOTS of back and forth about schedulers.  

    Where Windows falls down for me is that you have to be Mark Russinovich to diagnose these problems.  With Linux, everything is out there in the open, you have symbols and source for EVERYTHING (except Flash!), and you can track down the problem readily.

  136. Baughn says:

    I don’t really use windows much, so maybe I’m missing something obvious here, but wasn’t the real issue that programs are *allowed* to run at above normal priority?

    On linux, everything defaults to priority 0. You can lower the priority if you want, but only the superuser is allowed to *increase* it; thus, this sort of problem can’t happen. (There’s also the usual bunch of features like boosting the priority of inactive programs, etc.)

    Also, having an above-normal priority program hogging the CPU means normal-priority ones don’t get to run at all? Shouldn’t it be more of a proportional boost? Getting unquestioned access to the CPU is what realtime priority is for.

  137. Tom says:

    I’ve been frustrated with the Windows scheduler since NT. You can explain priorities to me all you want but I don’t see that as an excuse for something that normally takes a fraction of a second to take 30 seconds instead.

    I write multithreaded CPU-intensive scientific code. If I’m in Windows and run with 4 threads on my quad, the system becomes unusable. It takes 5-20 seconds to check my email, bring up a new browser window, etc. Ridiculous. My only solution is to run with 3 threads and wait longer for my job to finish–i.e., I’m less productive.

    If I run the same code with 4 threads under Linux or OS X, it’s as fast as it should be and I can barely tell it’s running. In fact, sometimes I literally forget that it’s running and hours later I’ll notice the louder CPU fan speed or _slightly_ sluggish browser performance and realize what’s been going on.

    I actually like WinXP quite a bit and use it for writing e-mail, surfing the web, etc., but because of the scheduler and certain other deficiencies it seems that Windows is unsuited to what I would call "real work."

  138. Huub van Dijk says:

    I had problems with ROXIO similar to those you encountered.

    – incompatiblility with AVG Free edition

    – AVG will refuse to update or install

     until ROXIO is updated

    – I did not manage the hugely complicated and

     possibly faulty registration process

     for my OEM installed ROXIO

    – I could not update ROXIO

    – ROXIO has a legion of autorunners

  139. Alexandre Grigoriev says:

    Rune,

    Unfortunately, Windows scheduler is no good against the runaway threads. And it’s very easy to detect those threads – they always use up their time slice. It’s very easy for the scheduler to give them dynamic priority bust (as opposed to boost).

    Even though Windows is supposed to give a priority boost to a foreground window, it doesn’t seem to help.

    And another strange thing of Windows scheduler: explorer file copy operation may take MUCH longer if you put its progress window to background. Why?

  140. B.Good says:

    The thought did cross my mind, "I wonder if he would have tried Firefox if he wasn’t now paid by Microsoft?"  That said, I realize this is a scheduler problem.  And possibly one that afflicts other operating systems based on some posts here.  However, can’t some of the best programmers on the planet figure out SOME way to keep the entire OS from becoming unresponsive?  If only to pop up a box that says "Program XYZ is consuming an unusually large amount of resources, would you like to reduce its priority to make your computer more responsive? (Yes|No)  Note that clicking ‘Yes’ may make Program XYZ run more slowly or take longer to complete."

  141. Neel says:

    Hi,

    Thanks for the article. I reached here after searching for defects that mentioned high cpu usage and I found

    https://bugs.adobe.com/jira/browse/FP-417

    Someone mentioned that flash programmer wrote an infinite loop.. absolutely.. I don’t see ANY other way this can happen. It has to be while (1).. grrrrrrr… I try to download multiple videos and watch later (because it’s much better than embedded ones)
    and when I start two IE instances, because of this flash problem, all I can do is watch the scrollbar until it completes the video downloading..

    This is a perfect oppurtunity for some millionaire to show his kindness and announce reward for any Flash programmer to find root cause of this.. I don’t see any other way this defect getting any attention otherwise.. (search for cpu usage related defects
    on adobe flash bugs page and see how many of them are happily unresolved..)

    Thanks,

    -Neel.

  142. Jason says:

    All of the pc’s I use at work and at home run XP, so I don’t really know about the problems with flash player on Vista..I’ve never used it!

    I have seen this CPU hogging problem with flash on XP, but only a few times.

    At my last job, I worked for a few years at a UK based e-learning company, developing (amongst other things) flash based elearning content which was delivered both online and offline. (I’m working as a C++ analyst/programmer nowadays at a CAD/CAM software company!)

    When our content was running in Internet Explorer or FireFox, (online or offline) the CPU usage wasn’t adversely affected.

    However, We did encounter the problem a few times during the initial development of the templates for our flash interface and content.

    But this was usually down to some poor logic or a bug in the actionscript (usually something stupid causing an infinite loop, or some complex, convoluted logic that was just generally screwing things up and causing hangups)..But we managed to fix the problems and the final flash software turned out to be brilliant!

    However, whilst browsing the web; I’ve visited a few sites recently which contained flash content which caused the CPU usage to jump alarmingly high and stay there until I navigated away from the offending page.

    However on the flip-side of that, I’ve also been on a lot of sites where the flash content has ran smoothly and without a glitch!

    So personally, I don’t believe it to be a problem with Adobe’s Flash browser plugins, I think it’s purely related to the people producing the flash content.

    It seems to be down to the flash developers Quality control processes (or lack thereof!).

    There is one flash related issue that’s been annoying me recently though:

    I’ve noticed that a lot of advertisers on various sites have posted debug builds of their flash adverts.

    The reason that’s annoying (other than the fact that they’re bloody adverts!) is because I use the debug version of flash player (as I still do a bit of flash development here and there myself) so whenever I encounter one of these debug builds in a web page, I keep  getting annoying popups asking where my local debugger is..Grrrr…I don’t want to debug their stupid advert!

    Again, proof of the point that some flash developers have no quality control systems in place…

    I will stress again, they seem to be in a minority as most flash content on the web is ok! (in my experience)

    So to my mind, Flash developers need to do a few simple things….

    1. Properly plan and design their projects..

    2. Code their projects more carefully to minimise/eliminate the risk of hangs/infinite loops/bugs.

    3. Properly plan the Debugging and testing of their projects, again to minimise the risk of bugs in their final release.

    4. Ensure that the final uploaded content is a Release build and not a bloody Debug build!

    All pretty fundamental stuff..Fundamental to programmers anyway!

    I suspect that most of the offenders in these cases are more artistic types with no formal programming background…That or just sloppy programmers!

    Anyway, that’s my 2 pence!

    Jas.

  143. Jason says:

    P.s. Great article by the way Mark.

    I’ve been using the sysinternals tools on and off for several years now and they’ve proved invaluable!

  144. we says:

    What you are all missing out on here is that it is not Flash fault:

    It is an OS DESIGN PROBLEM.

    Try this:

    Launch a browser with Flash or any media player on XP or Vista. You will see your other app using OpenGL or DirectX massively impacted.

    What a crappy OS Windows is..

  145. JMV says:

    I regularly do serious calculations on my system. If such a process runs in normal priority, like mathematica, such a system slows down a UI process by more than 50% on a single core system. The slow down can be by a factor of more than 20.

    WHY???????????????????

    Is it because a UI process switches threads constantly, and ends up effectively yielding all the time?

    Or is it because it makes kernel calls all the time?

  146. JMV says:

    To TOM:

    Knock down the priority of your scientific code on your quad!

    You’ll be relieved.

  147. Cliff says:

    When I was playing around with and reading about BeOS, one of the things that impressed me is that the UI thread *always* had priority; no matter what happened the UI *always* stayed responsive.  

    Am I misremembering this?

  148. TIm says:

    A wonderful article with excellent explanation of your trouble shooting approach. I already knew a little about Process Explorer, threads, dlls, etc. but this article puts it all together.

    Thank you Mark.

    -Tim

  149. Nathan Schultz says:

    When will process explorer have the ability to save the session. E.G. I would like process explorer run in the background and record the information to file for later playback. Is this possible? If not is that on the drawing board?

  150. molotov says:

    @Nathan Schultz

    Not quite the same as you appear to be inquiring about, but have you checked out Process Monitor?

    http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

  151. Andrew Janke says:

    Thanks for a good post, Mark. I find this sort of detailed forensic story very educational.

  152. Melvin says:

    Under Windows XP, I've seen Flash-based advertisements on one of the 'www.UsedEverywhere.com' sites that would monopolize the CPU, especially when the _same_ web-server injected a certain company's ad _twice_ on the same web-page. Navigating away from the web-page, or hitting "refresh" (to get different ads) is the usual bypass.

    As for schedulers, an IBM Fellow (named Wheeler) wrote a great scheduler for IBM mainframes, several decades ago! It used time-slicing, and classified each Virtual Machine into queues (Q0 for "trivial" processes, with Q1 and Q2 for "less-trivial" CPU-consumers, and Q3 for VM's (number crunchers) that always consumed 100% of their time-slice).  So, while the CPU typically ran at 100%, for hours on end, those in Q0 got sub-second response-time.  One had to query the "expansion-factor" to see how "over-committed" the CPU(s) were, i.e., an expansion-factor of '3' was an indication that IBM wants you to buy a 3-times-faster CPU, given that you have both an unlimited budget for such upgrades, and a justification from your CPE (computer performance evaluation) analyst that a faster CPU would produce greater "productivity" for your clients/users/customers.

    Some of those "dinosaur" mainframes used a SSD (solid-state device) as the primary paging-device,

    i.e., the OS would swap very-quickly to external RAM, thus avoiding rotational-delay of a disk-based paging-device. Now, decades later, Windows Vista has "ReadyBoost", to do the same.

    So, maybe Windows 8 will adopt the Wheeler Scheduler?  :-)

  153. Chris Quirke says:

    I'm also surprised that an app can hog the processor to that extent; I thought NT's pre-emptive multitasking was supposed to prevent that?

  154. Vish says:

    Hi Mark, Thanks a lot for this excellent article. I had same issue and I searched all over internet where people talking about different things without any concrete steps/results. Your article gave me an insight into using ProcessExplorer to find out more
    about which threads were being using by dllhost. I found that a .mp4 file I downloaded from internet was causing it. I logged in using Safe mode and deleted this file which resolved the issue. This article you wrote in 2008 still helped me in 2014! :-) Thanks
    again for a great article. Keep up the good work. – V