I PTE the fool: !SYSPTES 4 works in Vista SP1/WS08

System Page Table Entry (PTE) issues are some of the top support issues for servers that run large server applications and have a relatively large amount of Random Access Memory (RAM). PTEs are structures used to track pages of RAM, similar to the way a telephone number is used to track a telephone to a specific location.

You can now track down those 3f bugchecks using !sysptes using the public symbols. Usually when we hit a server running out of system PTEs, it will just tip over and we don't see an actual bugcheck.

Prior to Vista SP1/Windows 2008 if you tried to run SYSPTES 4 on a server with public symbols you'd get this error message: "Unable to get System PTE individual lock consumer information". Well Windows Dev has fixed the bug. Below is an example of what we would typically see, and then how we'd use !sysptes to narrow down who is consuming the space.

You find this sort of output in !VM:

0: kd> !vm 1
*** Virtual Memory Usage ***
Physical Memory: 999242 ( 3996968 Kb)
Page File: \??\C:\pagefile.sys
Current: 927744 Kb Free Space: 884312 Kb
Minimum: 927744 Kb Maximum: 927744 Kb
Page File: \??\E:\pagefile.sys
Current: 3072000 Kb Free Space: 3024624 Kb
Minimum: 3072000 Kb Maximum: 3072000 Kb
Available Pages: 265887 ( 1063548 Kb)
ResAvail Pages: 933615 ( 3734460 Kb)
Locked IO Pages: 1679 ( 6716 Kb)
Free System PTEs: 500 ( 2000 Kb)

********** Running out of system PTEs **************

******* 416179544 system PTE allocations have failed ******

Free NP PTEs: 1630 ( 6520 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 478 ( 1912 Kb)
Modified PF Pages: 477 ( 1908 Kb)
NonPagedPool Usage: 8814 ( 35256 Kb)
NonPagedPool Max: 32351 ( 129404 Kb)
PagedPool 0 Usage: 10590 ( 42360 Kb)
PagedPool 1 Usage: 994 ( 3976 Kb)
PagedPool 2 Usage: 958 ( 3832 Kb)
PagedPool 3 Usage: 972 ( 3888 Kb)
PagedPool 4 Usage: 931 ( 3724 Kb)
PagedPool Usage: 14445 ( 57780 Kb)
PagedPool Maximum: 54784 ( 219136 Kb)
Shared Commit: 4163 ( 16652 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 10930 ( 43720 Kb)
PagedPool Commit: 14485 ( 57940 Kb)
Driver Commit: 1963 ( 7852 Kb)
Committed pages: 747541 ( 2990164 Kb)
Commit limit: 1952440 ( 7809760 Kb)

So what now? If you don't have the registry value set below, well for all intents and purposes you SOL. So reboot, set the value, and then wait for repro. Usually when we see the issue it comes back pretty quickly as some driver is eating up the space.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management
trackptes REG_DWORD 0x1

Once the waiting is over and the system tips over, run !sysptes 4 and it will tell you what is allocating the PTEs and how many per call. The “!SYSPTES 4” command only lists driver PTE allocations. This is because, historically, drivers have made the most use and misuse of system PTEs. Sometimes you'll find one heavy hitter with a huge count (like in the article linked below), or in other instances you might find a certain sequence allocating many times, in either case you now have a clue as to who is using the PTEs and can either investigate that driver via break points, or contact the vendor who is eating up all the PTEs.

Cleaned up a bit for sanity's sake:

2: kd>!sysptes 4

VA MDL PageCount Caller/CallersCaller

f0769080 fce7fb18 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
efbbb8b8 fce0f658 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
f1c17080 fd0eb7a8 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
eff41820 fd41bc70 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
f1d10080 fcd91950 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
f027f108 fd051f88 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
efbf7080 fd7f3e80 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
f1cede10 fce71460 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
ef8a8080 fcedde80 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
ef963730 fc9c2868 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
f0281080 fccc52c0 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
f1dfaff8 fd156650 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
f0141080 fc6e82c0 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
ef8f3508 fd003a30 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b
ef637080 fee1dde0 2 mrxsmb+0x2bed2/mrxsmb+0x2da71
eff3fa18 fd304050 2 rdbss!RUserBuffer+0x2f/rdbss!UserBufferForLowIo+0x2b

The actual process of tracking down the PTEs is thoroughly explained here: Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues

So next time your system is acting up, get in there with Live KD and see what's going on!