Volume 6, Number 2

Copyright (C) 2004 Mark Russinovich

July 30, 2004 - in this issue:

- Dave Solomon Guest Article on Digging into Thread Details with Process

- Tools Updates
- Sysinternals is a Microsoft Windows XP Community Site

- Send us your success stories and win a Sysinternals T-shirt!

- PsExec
- The Memory Optimization Hoax

- San Francisco - September 27-October 1

- TechMentor
- Windows Connections
- Microsoft IT Forum

The Sysinternals Newsletter is sponsored by Winternals Software, on the Web at
http://www.winternals.com. Winternals provides
Intelligent Recovery for the Microsoft Enterprise.

The soon-to-be-released Windows XP Service Pack 2 will deliver numerous
benefits, but may result in unwanted or unforeseen behaviors
within the OS and within applications. Winternals Recovery Manager allows you to
safely and quickly roll back systems that are
adversely affected by patches and service packs, even if the systems have been
rendered unbootable. To learn more about Recovery
Manager, and to request an evaluation CD, please visit:


Dave Solomon and I are still hard at work on the next edition of "Inside Windows
2000" (to be called "Windows Internals", 4th
edition) and are expecting to have the manuscript complete in the next few
weeks. The book, which covers Windows 2000, Windows XP,
and Windows Server 2003, is showing growth of about 20% over the 3rd edition. We
think that you'll find the book even more valuable
than the previous edition because in addition to expanding the existing material
and adding new text to cover XP and 2003 changes,
we've added troubleshooting material for topics such as crash dump analysis,
system startup, memory, CPU, file system and the

Because I'm focusing my time on finishing the book this newsletter is brief, but
I'll continue regular newsletters once the book is
finished. In the meantime I'm including a guest article from Dave Solomon on
more advanced usage of Process Explorer than what I
covered in the last newsletter.

Enjoy and please pass the newsletter on to people you think will find it

- Mark Russinovich

1. Digging into Thread Activity with Process Explorer
By Dave Solomon (daves@..., http://www.solsem.com)

Process Explorer provides easy access to thread activity within a process. This
is especially important if you are trying to
determine why a process is running that is hosting multiple services (such as
Svchost.exe, Dllhost.exe, Inetinfo.exe, or the System
process) or why a process is hung.

To view the threads in a process, select a process and open the process
properties (double click on the process or click on the
Process->Properties menu item) and click on the Threads tab. This shows a list
of the threads in the process, the percentage of CPU
consumed (based on the refresh interval configured), the number of context
switches to the thread, and the thread start address. You
can sort by any of these three columns. As you select each thread in the list,
Process Explorer displays the thread ID, start time,
state, CPU time counters, number of context switches, and the base and current
priority. There is a Kill button which will terminate
an individual thread, but this should be used with extreme care.

New threads that are created are highlighted in green and threads that exit are
highlighted in red (the highlight duration can be
configured with the Options->Configure Highlighting menu item). This may be
helpful to discover unnecessary thread creation
occurring in a process (in general, threads should be created at process
startup, and not every time a request is processed inside a

The context switch delta represents the number of times that thread began
running in between the refreshes configured for Process
Explorer and is a different way to determine thread activity than the percentage
of CPU consumed, since many threads run for such a
short amount of time that they are seldom (if ever) the currently running thread
when the interval clock timer interrupt occurs and
hence are not charged for their CPU time.

The thread start address is displayed in the form "module!function", where
module is the name of the .EXE or .DLL. The function name
relies on access to symbol files for the module, which you get if you configure
Process Explorer to use the Microsoft Symbol Server
(see the help including with Microsoft Debugging Tools for Windows, which you
can download from Microsoft's web site at
http://www.microsoft.com/whdc/devtools/debugging/default.mspx). If you are
unsure what the module is, press the module button. This
opens an Explorer file properties window for the module containing the thread's
start address (e.g. the .EXE or .DLL). For threads
created by the Windows CreateThread function, Process Explorer displays the
function passed to CreateThread, not the actual thread
start function. That is because all Windows threads start at a common process or
thread startup wrapper function (BaseProcessStart
or BaseThreadStart in Kernel32.dll). If Process Explorer showed the actual start
address, most threads in processes would appear to
have started at the same address, which would not be helpful in trying to
understand what code the thread was executing.

However, the thread start address displayed may not be enough information to
pinpoint what the thread is doing and which component
within the process is responsible for the CPU consumed by the thread. This is
especially true if the thread start address is a
generic startup function (e.g. if the function name does not indicate what the
thread is actually doing). In this case, examining
the thread stack may answer the question. To view the stack for a thread, double
click on the thread of interest (or select it and
press the Stack button). Process Explorer displays the thread's stack (both user
and kernel, if the thread was in kernel mode).
While the user-mode debuggers (Windbg, Ntsd, and Cdb) permit you to attach to a
process and display the user stack for a thread,
Process Explorer shows both the user and kernel stack in one easy click of a
button. You can also examine user and kernel thread
stacks using Livekd from Sysinternals, however it is more difficult to use; note
that running Windbg in local kernel debugging mode,
which is only supported on Windows XP or Windows Server 2003, does not show
thread stacks.

[Personal note from Mark - this helped me solve why Powerpoint was hanging for
one minute every time I started it. I used Process
Explorer to look at the stack of the one thread in the Powerpoint process; it
was waiting on a call to connect to a network printer;
turns out I had a connection to a network printer that was not responding, and
because the Microsoft Office applications connect to
all configured printers at process startup, Powerpoint was "hung" until the
attempt to connect to the printer timed out. After I
deleted the connection to the printer, the problem went away.]


A number of tools have been updated since the last newsletter in April. Here's a
summary of enhancements:

* Process Explorer
Process Explorer is a Task Manager replacement (you can even replace Task
Manager altogether with a selection in Process Explorer's
Options menu).
- TCP/IP tab on process properties displays TCP and UDP connections; changes are
highlighted in color, making it easy to see new and
closed connections
- 64-bit version for AMD64 systems
- Support for setting process affinity masks on SMT (hyperthreaded) and SMP
- Display update performance enhancements

* DebugView
DebugView is a developer utility that lets you capture user-mode and kernel-mode
debug output without a debugger locally or across
the network.
- Larger kernel-mode and user buffers
- More highlighting filters
- Log file wrapping
- Support for Windows XP SP2 kernel debug output
- Clears output when it sees special "clear output" debug string

* Pendmoves
Pendmoves is a new Sysinternals utility that shows file move and delete commands
that have been scheduled on a system to take place
the next boot.

* Adrestore
Adrestore is a command-line utility that takes advantage of Windows Server 2003
Active Directory (AD) "tombstoning" to provide a
limited undelete capability for AD objects.

* Bluescreen Screen Saver
This screensaver, which mimics the blue-screen of death, now supports
multimonitor systems and Windows Server 2003.

* LogonSessions
This new command-line utility shows the list of logon sessions on a system,
including network, interactive and batch, and also shows
the processes running in each session.

* Pstools
The Pstools suite consists of 11 administrative command-line tools that work
locally and remotely. Many of them have been updated in
the past several months to include support for multiple computer systems that
you can specify on the command-line or in a text file.

* Autoruns
Autoruns shows the most comprehensive list of Windows registry and file system
locations that applications can use to configure
themselves to automatically start during the boot process.
- File system and registry locations now span multiple columns for easier
- Option to display only non-Microsoft entries, making it easy to see what
non-MS software is configured to start when you log in
- Can now disable an entry without deleting it from the registry (makes Autoruns
now a superset of Msconfig)
- Option to display services configured to start at boot time

In addition to tools updates, there's an update to a technical article:

- Boot.ini Option Reference
This reference now includes boot.ini switches added in Windows XP and Windows
Server 2003.

Sysinternals has been a community site for Windows XP Embedded at Microsoft.com
for several years and now I'm proud to announce that
Microsoft has also made Sysinternals a community site at the Windows XP Expert
Zone page:

Audiences in my seminars and conference presentations love to hear real-word
success stories, so if you have one troubleshooting
with the Sysinternals tools I'd love to hear about it. When you send one to me
at mark@... I'll enter you in a monthly
drawing to win a limited edition out-of-print Sysinternals t-shirt. Please be
as detailed as possible about how you used the
Sysinternals tool to solve the problem, and if possible and applicable send
screenshots and/or log files (Filemon and Regmon can
both save their output to a text file).


* Psexec
This article I wrote for the July issue is currently only available to
subscribers of Windows and .NET Magazine. It covers advanced
Psexec usage and describes how it works and interacts with Windows security.

* The Memory Optimization Hoax
In this Windows and .NET Magazine, which is available to non-subscribers,
article I describe the deceitful behavior of so-called
"RAM optimizers".

Sep 27- October 1, 2004 - San Francisco

For the first time open to the public, David Solomon and I are presenting the 5
day hands-on version of our Windows 2000/XP/2003
internals & advanced troubleshooting class in San Francisco, September
27-October 1, 2004. (Attendees must bring their own laptop;
configuration details will be supplied in advance--no purchase of additional
software is required).

This is the same class we teach to Microsoft employees around the world. It
covers the internals of processes & threads, thread
scheduling, memory management, security, the registry, and I/O system. Delve
into mechanisms such as system threads, system call
dispatching, interrupt handling, & startup & shutdown. Learn advanced
troubleshooting techniques using the Sysinternals tools and
how to perform crash dump analysis.

Why take this class? If you're an IT professionals deploying and supporting
Windows servers and workstations, you need to be able to
dig beneath the surface when things go wrong. Having understanding of the
internals of the Windows operating system and knowing how
to use advanced troubleshooting tools will help you deal with such problems and
understand system performance issues more
effectively. Understanding the internals can help programmers to better take
advantage of the Windows platform, as well as provide
advanced debugging techniques. And because the course was developed with full
access to the Windows kernel source code and
developers, you know you're getting the real story.

For details and registration, visit


After speaking at Microsoft TechEd US and TechEd Europe in May and June, I'm
enjoying the summer at home in sunny Texas. Here are
my next three conference speaking engagements:

September 27-October 1, 2004 San Jose, CA

I'm delivering four sessions at this conference, all on September 29, including
"Windows and Linux: A Tale of Two Kernels" as a
keynote. The other sessions I'm presenting are "Windows Hang and Crash Dump
Analysis", "Windows XP and Windows Server 2003 Kernel
Changes", and "Troubleshooting Windows Boot and Startup".

You can read my abstracts and find a link to the conference registration page at

October 24-27, 2004
Orlando, FL

At this conference I'm delivering my "Troubleshooting Windows Boot and Startup"
talk as a general session and am also presenting
"Troubleshooting Windows with the Sysinternals Tools", both on the 24th (I'm
proud to say that Microsoft Network - MSN - has invited
me to present as the key-note address a day-long Windows troubleshooting session
at their annual MSN Engineering Excellence
Conference that week in Seattle).

Read the abstracts and go to the conference site from

Copenhagen, Denmark

I'm delivering a day-long pre-conference tutorial session with Dave Solomon on
Windows core security internals as well as several
breakout sessions on my own that are yet to be determined. You can find the
pre-conference tutorial abstract and registration
information here: http://www.microsoft.com/europe/msitforum/

Thank you for reading the Sysinternals Newsletter.

Skip to main content