Volume 3, Number 2

Copyright (C) 2001 Mark Russinovich

August 20, 2001 - In this issue:


- Process Explorer v5.1
- Handle v2.0
- PsInfo v1.1
- PsExec v1.24
- PsLogList v2.07
- Windows XP Source Layout
- DebugView v4.13
- PageDefrag v2.1, Contig v1.41
- Sysinternals at Microsoft

- Inside Windows 2000, The Interactive DVD
- Mark the date: Russinovich and Solomon teach together in Austin
- Command-line disk management
- WINHEC 2001 slides online
- New file system filter driver interfaces in XP
- Crash Windows with a key stroke
- Windows XP Prefetching
- Windows .NET Connections

My Itanium Experiences

The Sysinternals Newsletter is sponsored by Winternals Software, on the Web at
http://www.winternals.com. Winternals Software is the leading developer and provider of advanced systems tools for Windows NT/2K/XP. Winternals Software products include FAT32 for Windows NT 4.0, NTFSDOS Professional Edition (a read/write NTFS driver for DOS), and Remote Recover.

Winternals is proud to announce Defrag Commander version 1.31, the fastest, most thorough enterprise-defragmenter available. Now you can manage defragmentation schedules across your entire Windows enterprise from a simple MMC snapin - without even having to install any client software on your NT or Windows 2000 systems. A 10-system license is available for on-line purchase for only $169, and aggressive quantity discounts are available. Visit
http://www.winternals.com/39 for more information or to download and use free for 30 days. ~~~~~~~~~~~~~~~~~~~~~~~~~

Hello everyone,

Welcome to the Sysinternals newsletter. The newsletter currently has 33,000 subscribers.

I use computers in several different capacities: as a software developer; as a user; and as a systems administrator (for my own network of development and test systems).  Because I want to reach the widest audience for Sysinternals freeware tools and Winternals commercial products, most utilities target at a minimum Windows NT 4 through Windows XP, and many also run on Windows 95 through Windows Me. The majority of the tools I write are systems-level, meaning that they often contain special-case code for one or more of the various Windows variants. Unfortunately, there are significant user-interface and kernel differences between the Windows NT line and the Windows 9x line, and even differences between versions from the same line.

I used to perform testing by having several debug systems, all configured to multiboot the Windows flavors that I’ve enumerated, and booting into a version, testing, and then rebooting into the next. Besides being slow, testing would sometimes be skewed by interfering droppings of intermediate test builds of utilities that I hadn’t cleaned up properly. While I still have a collection of multibooting systems for debugging problems, an innovative product called VMWare (www.vmware.com) has made my testing cycles faster and easier.

VMWare is a virtual machine environment that you use to create virtual computers (guests) that run on top of Windows NT, Windows 2000, or Linux installations (hosts). A guest can run most variants of Windows, DOS, and Linux. Its disconcerting to see a version of an operating system run on top of another one, especially when you make a virtual machine full screen – there’s virtually no way to detect that you aren’t sitting at the screen of a Linux computer instead of a Linux computer’s virtual display running on top of Windows 2000, for example . A virtual machine is a software-imposed box that encapsulates the operating system and applications running in the virtual machine so that when they try and interact with the hardware they believe is present, the virtual machine software intercepts the accesses. The virtual machine software creates virtual devices that serve as carefully controlled surrogates to real hardware or virtual hardware. For example, when an application that’s running in a virtual machine that is being displayed in a window writes to the virtual machine’s display memory, the virtual machine software updates the contents of the window that represents the virtual machine’s display instead of letting the accesses touch the real video memory. If the virtual machine were running in full-screen mode then the virtual machine software would let such accesses manipulate video memory directly.

The virtual machine capability alone makes VMWare useful, since you can have any number of “test systems” lying around your hard drive. What makes VMWare even more powerful is the fact that you can “suspend” a virtual computer, saving its state to disk, and then later resume it, within seconds restoring the state it had when you suspended. But that’s not all. I make extensive use of a feature called “non-persistent disks”, which has VMWare start a virtual computer with its virtual disks containing the contents you’ve defined (for instance, with just a basic OS installation) and then track changes so that you can undo them and revert back to the starting state. If I install test versions of a utility and then need to make sure that I’m back at a clean state that doesn’t reflect any of the utility’s changes, I simply undo the changes. I also find VMWare useful in my user and systems administrator roles. Whenever I download an application that I’m not sure I want to keep, rather than possibly compromise my development system or fill it with left-behind debris when I uninstall an application, I simply try it in a virtual machine with a non-persistent disk first.

There are some relatively minor drawbacks to the current version that have prevented me from using my real debug computers as much as I do. The first is that, because the virtual machines do not support DirectX, you can’t run Numega’s SoftICE debugger except with the virtual machine at a screen resolution of 640x480. The second is that VMWare’s virtualization of serial ports is limited in a way that prevents kernel debugging using Windbg or Kd either between the host and a guest or between guests. The latter issue should be addressed in the next major VMWare release, however.

What’s particularly interesting about VMWare is that, up until VMWare demonstrated otherwise, computer scientists predominantly held the belief that it was impossible to virtualize the x86 architecture in software well enough to run an unmodified operating system without prohibitive performance degradation. The challenges are significant, particularly in the area of hardware device virtualization, and VMWare’s success is seen as a breakthrough - so much so that VMWare recently published a “best of conference” paper on how they implemented device virtualization at the annual USENIX Technical Conference. You can read the paper at

Speaking of the USENIX Technical Conference, I also coauthored a paper published there, High-Performance Memory-Based Web Servers: Kernel and User-Space Performance (
http://www.sysinternals.com/files/webserver.pdf). The paper describes some of the innovative research I participated in while working at IBM Research. The contributions of the work are in the area of kernel-mode Web-server acceleration, and besides contributing to the architecture, I was responsible for much of the implementation on Windows NT and Windows 2000. The project was so successful that we consistently held SPECWeb world records for web-serving performance, its been released as a still-evolving IBM product called IBM Netfinity Web Server Accelerator, and both Microsoft and the Linux community have incorporated several of the key ideas into their own products (such as IIS for Windows 2000 and Tux on Linux).

Please pass the newsletter to friends you think might be interested in its content.



========== WHAT'S NEW AT SYSINTERNALS ==========
Process Explorer is the new name for HandleEx, a multipurpose utility that shows you detailed information about processes, the DLLs they have loaded, and the handles to operating system resources they have open. Along with its new name, the latest versions of Process Explorer introduce a slew of new features including efficient auto-refresh, a CPU usage column, the ability to change process priorities, and reporting of process session IDs if the system is a terminal server. Plus, it shows much more information about processes. For example, you can view the list of services running within a process (if applicable), examine a process’ environment variables, and view process command lines. Using a new sorting option called “process tree”, you can graphically see the parent-child relationships of processes, something that can help you identify a process’ purpose. Process Explorer runs on all Windows 9x versions as well as Windows NT 4 through Windows XP.

Something that may not be immediately obvious from Process Explorer’s interface is its ability to show you what processes have a particular file or directory open. Switch the display to handle mode and enter the name of the file or directory in question into the search dialog. The search feature is most often used to track down the process that’s preventing you from deleting or renaming a file or directory, but you can use it to list processes that have specific DLLs loaded as well.

Download Process Explorer v5.1 at

Handle is the command-line version of Process Explorer’s handle-viewing functionality that allows you to see the handles to operating system resources, such as files, that processes have open. In some cases using Handle is a faster way to identify a process that has a file or directory open than using Process Explorer’s search facility, because you simply enter a part of the file or directory’s name on the Handle command line. Besides more efficient lookup of handle names, Handle 2.0 now works on Windows 95/98/Me.

Download Handle v2.0 at

The latest addition to the PsTools suite of administrative tools is PsInfo. PsInfo is a command-line tool that reports a computer’s hardware and operating system characteristics. For example, it tells you what operating system is running, including the service pack number, the install and expiration date (if applicable), and configuration (e.g. domain controller, member server). It also lists the type, speed and number of CPUs on the computer, and the amount of physical memory installed. This information makes PsInfo handy for system identification or inventory. Like all of the tools in the PsTools suite, PsInfo runs on the local or a remote Windows NT, 2000 or XP computer.

Download PsInfo v1.1 at

PsExec allows you to launch processes on remote systems without having to manually install any software on the remote system, as long as the system is in your Network Neighborhood. PsExec’s strength lies in its support for redirected console program I/O, where it handles a remote console program’s keyboard input and text output as if it were running on your local system.

The latest version of PsExec supports multiple running processes on the same remote system, allowing you to remotely open several command prompts on another system, for example. It also has a new “force copy” option that lets you specify that you want PsExec to copy a file to a remote computer for execution, replacing any existing version.

Download PsExec v1.24 at

PsLogList, another PsTools utility, dumps event logs from the local or a remote system. Unlike eloglist, the Windows 2000 Resource Kit tool with the same purpose, PsLogList displays a record’s full event string, and when dumping remote logs it uses the event log string-resource files from the remote system instead of from the local one.

Version 2.07 of PsLogList includes a command-line switch so that you can direct it to dump extended event log record information (the additional information that can be associated with a record), and a new switch so that you can view the records from only the specified most recent days.

Download PsLogList v2.07 at

Have you ever wondered how Microsoft organizes the source code that makes up the Windows NT/2000/XP kernel? I did, too, and realized that the information is publicly available. Whenever Microsoft releases a new build of Windows 2000/XP it releases several versions:

• a uniprocessor “free” version
• a uiniprocessor “free” version that supports more than 4 GB of physical memory
• a multiprocessor “free” version
• a multiprocessor “free” version that supports more than 4 GB of physical memory
• a “checked” multiprocessor version
a “checked” multiprocessor version that supports more than 4 GB of physical memory

For a total of 6 versions. What distinguishes the “free” from the “checked” is not the fact that the “free” version is free (its not), but that the “checked” version, which is included in the MSDN CD set, includes code and data that help with operating system and device driver debugging. The “free” version is simply compiled free of the extra code that is included within conditional compilation statements.

Examples of the extra checked code include debug print statements that report what the operating system is doing, more rigorous consistency checking on the parameters passed into kernel-mode functions from device drivers, and “assertion” statements. An assert statement validates a developer’s assumption about conditions that apply to an area of code. For example, if a pointer should point to a data structure with an identifying signature, a developer could insert an assertion to that effect. What happens when the assertion’s assumption is violated? That depends on the assertion, but in the case of those included in the checked builds a debug output message is printed and then a debugger breakpoint is triggered so that the kernel debugger activates (or, if no debugger is active, a crash occurs that hopefully generates a dump file that can be later examined).

In order to make identifying the problems caught by assertions easy, the assertions used in the checked build print out the file, function, and line number of the assertion’s location. I’m able to show the source tree layout by extracting these strings with my “strings” string-extraction utility and organizing them into a java treeview.

View the Windows XP source layout and learn more about assertions at
http://www.sysinternals.com/ntw2k/info/xpsrctree.shtml (note that the IE will display a warning about the page loading slowly. Dismiss it and you’ll see the tree).

DebugView is a developer utility that lets you capture debug output from applications or drivers on the local system or a remote one - even from multiple systems simultaneously. This latest release adds compatibility with Windows XP RC 1. DebugView works on Windows 9x, Windows Me, Windows NT, Windows 2000, and Windows XP.

Download DebugView v4.13 at

Ranking up at the top of the most popular Sysinternals downloads are PageDefrag and Contig. PageDefrag defragments core system data files at boot time and Contig is a command-line file defragmentation utility. The latest versions of these tools have an improved defragmentation engine for Windows 2000 and each has some of its own enhancements.

Besides defragmenting Registry and paging files, the latest version of PageDefrag also defragments event log files. And similar to Chkdsk in Windows 2000, PageDefrag now has a 3-second count down during the boot process during which you can press any key to have it skip defragmentation.

What makes Contig unique is the fact that you can use it to defragment individual files, entire directories, or an entire disk. Contig is aimed at optimizing particular files that are critical to application performance because, unlike commercial defragmenters, it does not consolidate free space in order to help prevent future fragmentation on a disk. Besides the enhanced defrag engine, the new Contig has better output formatting both in non-verbose and verbose modes.

Download PageDefrag v2.1 at
Download Contig v1.4 at

Once again here’s the latest installment of Sysinternals references in Microsoft Knowledge Base (KB) articles released since the last newsletter. This brings to 27 the total number of KB references to Sysinternals.

0x8000FFFF "Catastrophic failure" Message with SQL Server ODBC Driver

ACC: Error Message: ActiveX Component Can't Create Object http://support.microsoft.com/support/kb/articles/Q296/2/05.ASP

HOWTO: Determine the Version of MSXML Used by Internet Explorer http://support.microsoft.com/support/kb/articles/Q296/6/47.ASP

HOWTO: Troubleshoot "ADODB.Connection" Error 800a0bb9 from Recordset DTC http://support.microsoft.com/support/kb/articles/Q197/3/23.ASP

INFO: Troubleshooting Guide for 80004005 and Other Error Messages http://support.microsoft.com/support/kb/articles/Q183/0/60.ASP

XADM: Event ID 3036 and 3026 Messages Occur When You Populate Mailbox Store http://support.microsoft.com/support/kb/articles/Q296/1/36.ASP

================== INTERNALS INFORMATION =============
Dave Solomon and I wrote “Inside Windows 2000, 3rd Edition”, the official guide to the internals and architecture of Windows 2000, and we’re excited to announce the impending release of the “Inside Windows 2000” DVD tutorial. The tutorial actually consists of 5 DVDs with over 9 hours of content consisting of Dave and me teaching you the inner workings of Windows 2000 in an informal setting.

Topics covered include memory management, processes and thread, storage, file systems, networking, and more, and each is broken into 10-20 minute modules for convenient viewing. The presentation includes dozens of diagrams, demos, screenshots, and summary lists, and each module concludes with review questions to test and solidify your understanding.

Final pricing and the release date have not yet been finalized, but if you wish to be notified of the release send e-mail to
mailto:video@... with the word “interest” in the subject. Or, watch http://www.sysinternals.com/video/ for details.

*MARK THE DATE: RUSSINOVICH & SOLOMON TEACH TOGETHER IN AUSTIN If you’re interested in the DVD-set you’ll definitely be interested in an opportunity to see Dave and me teach NT internals live. Come to Austin, TX December 11-13 to hear us present our 3-day Windows XP/2000/NT Internal Architecture class. The class is based on “Inside Windows 2000, 3rd Edition” and covers environment subsystems, system call dispatching, system threads, startup & shutdown, registry internals, processes and thread scheduling, memory management, security, the I/O system, storage, NTFS, and the cache manager. By understanding the inner workings of Windows XP & 2000 you can take advantage of the platform more effectively and more effectively debug and troubleshoot problems.

When details are available we’ll let you know in the newsletter and on the web site, so stay tuned!

Bowing to pressure from systems administrators, Microsoft has finally released a scriptable command-line tool for managing disks on Windows 2000. DiskPart lets you create volumes or partitions, create and break mirrors, extend volumes, and examine details of the disks, volumes and partitions. What’s nice is that DiskPart is a free download from

If you missed Microsoft’s annual Windows Hardware Engineering Conference (WINHEC) you can still view the slides from many of the presentations. Visit
http://www.microsoft.com/winhec/sessions/driver.htm for a list of sessions, descriptions and accompanying links to Power Point decks. Presentations of general interest are ones on driver debugging techniques, filter drivers, and NDIS miniport and intermediate drivers.

If you have a real-time file mirroring, on-access virus checking, or hierarchical storage management product that’s based on a file system filter driver then you’ll want to make sure that your abreast of the changes Microsoft has introduced in Windows XP that affect you.

The biggest change relates to fast I/O routines, the special functions that a file system driver registers so that the Memory Manager, Cache Manager, and I/O system can execute file system I/O and interact with file system drivers without having to generate IRPs. It may come as a surprise to some of you to find out that there are 6 fast I/O routines for which file system filter drivers are always bypassed. They are:


The Memory Manager calls FastIoAcquireFileExclusive and FastIoReleaseFile before and after creating a section backed by a file, and other kernel-mode subsystems can call these routines to temporarily prevent sections from being created. The Cache Manager calls FastIoAcquireFileForCcFlush and FastIoReleaseFileForCcFlush before and after flushing all or part of a file’s modified cached data back to disk, and the Memory Manager calls FastIoAcquireFileForModWrite and FastIoReleaseFileForModWrite before and after writing dirty mapped-file pages back to a file.

Rather than invoke fast I/O functions directly, kernel-mode subsystems use surrogate file system runtime (FsRtl) routines. The runtime routines for most fast I/O functions invoke filter drivers by getting a reference to their filter device object via a call to IoGetRelatedDeviceObject on the target file object and then calling the filter’s corresponding fast I/O routine, but runtime routines related to the fast I/O calls just listed instead call IoGetBaseFileSystemDeviceObject, which returns the underlying file system driver’s device object. The reason that the runtime bypasses filters for these functions is, in my opinion, a little lame: the runtime just doesn’t trust filter drivers. If a filter doesn’t pass these calls down to the underlying file system driver it causes file system data corruption and almost certainly a crash. But there are a lot of things that a filter driver can do to cause crashes.

In Windows XP the FsRtl introduces a new function, FsRtlRegisterFileSystemFilterCallbacks, that filter drivers use to register callbacks for these various operations. This makes it possible for file system filter drivers to examine these operations and even to fail them, and the runtime can ensure that the underlying file system driver is always invoked when appropriate. Where can you find the documentation for the function? The Windows XP Installable File System Kit, which is available for $995 from Microsoft:

A few newsletters ago I told you how you could add a setting to the Windows 2000 Registry that enables you to initiate a Bluescreen crash of a running system with a special key sequence, allowing you to analyze systems that are otherwise not responding to input. How about a way to crash Windows NT and Windows 2000 without having to set anything in the Registry?

Open a command-prompt window, change the current directory to the root of your boot drive (the drive with \winnt on it), and type “dir /s”. While that’s running type F7+Enter a few times and then abort the listing by entering Ctrl-C. You’ll either immediately encounter a blue screen or a spontaneous reboot. Cool, huh? Unfortunately, unlike the trick that I previously reported, this one isn’t designed behavior, but rather a bug for which you can find a report at

For those of you that missed the newsletter that included the keystroke-crash setting, add the following Registry value to your Windows 2000 or XP Registry as per the instructions in the Microsoft Debugging Tools help file:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\i8042prt\Parameters\CrashOnCtrlScrll DWORD 1

Reboot and you can crash the system by typing scroll-lock twice while holding down the left-control key.

Microsoft has focused Windows XP on improving the end-user experience, and users consider a big part of the experience the speed at which a computer turns on and is usable. Therefore, a Microsoft developers have spent a great deal of effort on improving the performance of the boot process and application startup. They’ve addressed this in several ways: the first is that serial and networking device drivers initialize in parallel, unlike in Windows 2000 where they initialize serially. Second, Winlogon no longer waits for the workstation service (which itself waits on networking services) to become available before presenting the logon dialog and allowing a user to logon. Finally, they’ve incorporated a technique called “prefetching” into the boot process and application startup. I’m going to briefly cover how prefetching works and point you at a whitepaper Microsoft published on boot optimizations.

Windows (all versions except for real-mode Win3.1) is a demand-paged operating system, where file data and code is “faulted” into memory from disk as an application attempts to access it. Data and code is faulted in “page”-granular chunks where a page’s size is dictated by the CPU’s memory management hardware. A page is 4 KB on the x86. Prefetching is the process of bringing data and code pages into memory from disk before it’s demanded.

In order to know what it should prefetch, the XP Cache Manager monitors the page faults that applications incur as they startup. By default it traces the first 2 minutes of the boot process and the first 10 seconds of application startup. After collecting a trace that’s organized into faults taken on the NTFS Master File Table (MFT) metadata file (if the application accesses files or directories on NTFS volumes), the files referenced, and the directories referenced, it notifies the prefetch component of the Task Scheduler by signaling a named event object. The Task Scheduler performs a call to NtQuerySystemInformation with an information type that specifies a query for a trace and reads it in. After performing post processing on the trace data the Task Scheduler writes it out to a file under \Windows\Prefetch. The file’s name is the name of the application to which the trace applies followed by a dash and the hexadecimal representation of a hash of the file’s path. The file has a “.pf” extension so an example would be NOTEPAD.EXE-AF43252301.PF.

An exception to the file name rule is the file that stores the boot’s trace, which is always named NTOSBOOT-B00DFAAD.PF (a convolution of the hexadecimal-compatible word “BAADF00D”, which programmers often use to represent uninitialized data). Any faults taken in the System or Idle processes are considered part of the boot, which makes sense since it’s in the System process that device drivers load and initialize.

When the boot starts or an application executes the Process Manager calls the Cache Manager to give it an opportunity to perform prefetching. The Cache Manager looks in the prefetch directory to see if a trace file exists for the prefetch scenario in question. When one does the Cache Manager calls NTFS to prefetch any MFT metadata file references, it reads in the contents of each of the directories referenced, and finally it opens each file referenced and uses the Memory Manager to read in any data and code specified in the trace that’s not already in memory. The Memory Manager initiates all of the reads asynchronously and then waits for them to complete before letting an application’s startup continue.

How does this scheme provide a performance benefit? The answer lies in the fact that during normal boot or application startup the order of faults happens such that some pages are brought in from one part of a file, then another, then a few from a different file, then some from a directory and so on. This jumping around corresponds directly to jumping around on the disk, and Microsoft has learned through analysis that disk seek times are a dominant factor slowing boot and application startup times. The seeking is virtually eliminated when prefetching reads in data from a particular file or directory all at once before moving on to another.

To minimize seeking even further, every three days or so the Task Scheduler organizes a list of files and directories in the order that they are referenced during a boot or application start and stores the list in a file named \Windows\Prefech\Layout.ini. Then it launches the system defragmenter with a command-line option that tells the defragmenter to defragment based on the contents of the file instead of performing a full defrag. The defragmenter finds a contiguous area on a volume large enough to hold all the listed files and directories and then moves them into the area so that they are stored one after the other. 

These are the basics of the XP prefetching scheme, which Microsoft reports speeds the boot process significantly (you can actually see that XP boots much faster than Windows 2000). You can find some more information on prefetching and the other fast-boot enhancements in a white paper at
http://www.microsoft.com/hwdev/fastboot/. Also, Dave Solomon and I have begun work on the XP-revision of “Inside Windows 2000” (for publication in mid-Spring), and you’ll find an even more detailed description there.

Those of you that missed me at TechEd can hear me speak at the Windows .NET Connections conference in Scottsdale, Arizona on October 3. I’m delivering presentations on Windows NT/2000 crash dump analysis and kernel changes in Windows XP. Other speakers at the conference include fellow Windows 2000 Magazine contributing editors Mark Minasi and Sean Daily. Let me know if you attend that you heard about the conference from the newsletter.

You can see the abstracts to my talks and find a link to the Windows .NET Connections site at

================ WHAT'S COMING UP =================
Microsoft has loaned me an Itanium system so that I can port Sysinternals utilities to Win64. The machine has some impressive specs, by the way: 2 733 MHz processors and 8 GB (!) of RAM. Next time I’ll tell you about my porting experiences, including the changes I’ve had to make to various utilities so that they work on Win64.

Thank you for reading the Sysinternals Newsletter.

Skip to main content