Symptoms: Lack of Free System Page Table Entries (PTEs) and system-wide delays (I/O request failures)


Introduction


The purpose of this article is to provide prescriptive guidance on how to troubleshoot free system page table entries (PTEs) in regards to Windows performance analysis.


Start with the following performance counters to analyze free system PTE’s:


· \Memory\Free System Page Table Entries


A page table is the data structure used by the Windows Virtual Memory Manager (VMM) to store the mapping between virtual addresses and physical addresses in memory. The performance counter Free System Page Table Entries is the number of page table entries not currently used by the system.


From the process perspective, each element of virtual address conceptually refers to a byte of physical memory. It is the responsibility of the Virtual Memory Manager (VMM) in conjunction with processor memory management unit (MMU) to translate or map each virtual address into a corresponding physical address.


The VMM performs the mapping by dividing the RAM into fixed-size page frames, creating system PTEs to store information about these page frames, and mapping them. System PTEs are small kernel-mode buffers of memory that are used to communicate with the disk I/O subsystem and the network. Each PTE represents a page frame and contains information necessary for the VMM to locate a page.


Note: Troubleshooting System PTE’s is explained in more detail at in the “Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues” article mentioned in the References section below.


This article is grouped by symptoms, then by possible causes.


Symptoms: Lack of Free System Page Table Entries (PTEs) and system-wide delays (I/O request failures)


Applies to:



  • 32-bit Windows Server 2003 (all editions) unless otherwise specified
  • 32-bit Windows XP (all editions) unless otherwise specified
  • 32-bit Windows Server 2000 (all editions) unless otherwise specified

Symptom Details:


· Lack of Free System Page Table Entries (PTEs): Use the “Memory\Free System Page Table Entries” performance counter for values under 5000. Alternatively, the !pte command in the kernel debugger can be used to examine PTEs.



  • Periodic system-wide delays: System-wide delays or hangs that occur regularly or occur during elevated load on the system. This is measured by the user experience and I/O response times of the system.


Possible Cause: Use of Physical Address Extension (PAE) Kernel


How to Diagnose


· To determine if the Windows 2003 server is booted with the PAE kernel, by checking for a value of 1 in the registry key, “HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PhysicalAddressExtension”. Alternatively, the boot.ini file can be searched for the PAE boot switch.
Note: when execution protection is enabled on 32-bit Windows, the system automatically boots in PAE mode (automatically selecting the PAE kernel).


· The PAE switch allows the Windows operating system to address more than 4GB’s of physical memory and in certain environments the PAE switch is automatically enabled. On x86 systems, the page table index is 10 bits wide (9 on PAE), allowing you to reference up to 1024 4-byte PTEs. Therefore, the /PAE switch causes PTE entries to use twice the normal allotted virtual address space. For more information, see “Microsoft Windows Internals” by Mark E. Russinovich and David Soloman.


Possible Solutions and/or Recommendations


· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see http://support.microsoft.com/kb/294418.


· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see http://support.microsoft.com/kb/294418.


· Remove the /PAE switch: If not needed, then remove the /PAE switch from the boot.ini file.



Possible Cause: Use of the /3GB switch


How to Diagnose


· The /3GB switch significantly reduces the number of free system PTE’s. To check if the /3GB switch is enabled use one of the following methods:


o Open the boot.ini file and look for the /3GB switch on the most recently used boot option.


o Check the value of the registry key: “HKLM\SYSTEM\CurrentControlSet\Control\SystemStartOptions”


o Check the value of the WMI property root\cimv2\Win32_OperatingSystem.MaxProcessMemorySize for a value close to 3,145,536 (3GBs) and greater than 2,097,024 (2GBs).


· Refer to the chart below for the starting amount of free PTE’s in different operating system configurations. Note the different in the amount of free PTE’s when the /3GB switch is enabled.


 


Possible Solutions and/or Recommendations


· Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” http://support.microsoft.com/kb/316739.
Note: For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch. For more information, go to http://technet2.microsoft.com/windowsserver/en/library/c5ccbaec-f552-4f61-a488-8ee3330d1eeb1033.mspx.


· Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.


· Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.


· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see http://support.microsoft.com/kb/294418.


· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see http://support.microsoft.com/kb/294418.


 


Possible Cause: Too much physical memory


How to Diagnose


· Refer to the “Estimated Kernel Resources Chart” in the More Information section for the starting amount of free PTE’s in different operating system configurations. Note the how increasing the amount of physical memory on the server reduces the number of free PTE’s on the server.


 


Possible Solutions and/or Recommendations


· Reduce the amount of physical memory: Reduce the amount of physical memory to free up more virtual memory for system PTE’s.


· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see http://support.microsoft.com/kb/294418.


· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see http://support.microsoft.com/kb/294418.


 


 


Possible Cause: High resource consumption and/or poorly written device drivers


How to Diagnose


 


Possible Solutions and/or Recommendations


· Remove any unnecessary third-party drivers and drivers not listed on the Hardware Compatibility List (HCL): Drivers that are not on the Hardware Compatibility List (HCL) have a higher likelihood of causing system problems and/or misuse of system resources. For more information about the Microsoft Hardware Compatibility List (HCL) go to, http://winqual.microsoft.com/hcl/.


· Use /BASEVIDEO: Use the /BASEVIDEO boot switch in the boot.ini file or a generic video driver to free up system page table entries. Video boards use the system page table entries to map their buffers in kernel space. This usage competes with the need for system page table entries by Microsoft Exchange.


· Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” http://support.microsoft.com/kb/316739. For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch.


· Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.


· Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.


· Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see http://support.microsoft.com/kb/294418.


· Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, go to http://msdn2.microsoft.com/en-us/library/bb870880(VS.85).aspx.


 


 


Possible Cause: High number of mailboxes on Microsoft Exchange Server


How to Diagnose


 


Possible Solutions and/or Recommendations


· Reduce the number of mailboxes on the server and/or remove the public folder role from the mailbox server.


High number of Microsoft Exchange mailbox stores that have the public folder server set as their default public store.


· Reduce the number of mailbox stores that have the public folder server set as their default public store. This action will reduce the number of clients (and, as a result, the number of user sessions) on the public folder server.


Estimated Kernel Resources Chart


This chart shows the estimated, initial maximum size of kernel resources on 32-bit Windows Server 2003 computers when using PAE, /3GB, with booted different physical memory sizes. Use the “!vm” command when doing a kernel debug to find the actual kernel resources.


  







































Memory


Default                            ( /PAE for 6-16GB )


/3GB


1GB


Free System PTE: 51k


         Paged Pool: 282MB


 Non Paged Pool: 212MB


Free System PTE: 32k


         Paged Pool: 163MB


 Non Paged Pool: 131MB


2GB


Free System PTE: 196k


         Paged Pool: 360MB


 Non Paged Pool: 262MB


Free System PTE: 16k


         Paged Pool: 262MB


 Non Paged Pool: 131MB


3GB


Free System PTE: 195k


         Paged Pool: 360MB


 Non Paged Pool: 262MB


Free System PTE: 14k


         Paged Pool: 262MB


 Non Paged Pool: 131MB


4GB


Free System PTE: 106k


         Paged Pool: 336MB


 Non Paged Pool: 285MB


Free System PTE: 15k


         Paged Pool: 258MB


 Non Paged Pool: 154MB


6GB


Free System PTE: 186k


         Paged Pool: 366MB


 Non Paged Pool: 262MB


Free System PTE: 12k


         Paged Pool: 239MB


 Non Paged Pool: 131MB


8GB


Free System PTE: 182k


         Paged Pool: 366MB


 Non Paged Pool: 262MB


Free System PTE: 12k


         Paged Pool: 225MB


 Non Paged Pool: 131MB


12GB


Free System PTE: 175k


         Paged Pool: 366MB


 Non Paged Pool: 262MB


Free System PTE: 12k


         Paged Pool: 196MB


 Non Paged Pool: 131MB


16GB


Free System PTE: 167k


         Paged Pool: 366MB


 Non Paged Pool: 262MB


Free System PTE: 12k


         Paged Pool: 169MB


 Non Paged Pool: 131MB





 


 


 


 


 


 


 


 


 



  


  


  


  


  


  


  


  


  


  


  


References



Contributors:


Clint Huffman, Shane Creamer, Rick Anderson, Maximilian Silva, Matthew Walker, Pavel Lebedynskiy, John Rodriguez, Mike Lagase, Yong Rhee


Comments (5)

  1. Anonymous says:

    It was a dark and stormy night… actually it was sunny and clear, but just thought I’d start this blog

  2. Omkar says:

    Hello………This article is just wonderful and gives detail steps about how to go troubleshooting this problem of low on system page table entries.

    Thanks a lot for this wonderful article. Gives simple and awesome explanation.

  3. Omkar says:

    Hello…wanted to clarify one thing about too much physical memory…you have mentioned that too much memory can cause  low system page table entries problem , so wanted to know how efficient is it to reduce physical memory.

    Also I understand that it’s not only physical memory but there could other heavy consumer of pte’s like video card.

    In such we could enable this registry entry to find who is the highest consumer or not releasing the pte. http://support.microsoft.com/kb/256004

    But I wanted to get a  general ideal about how efficiently pte stand at a high number on machines with huge amount of physical memory.

    Specially in scenario where there is huge amount of physical memory on data centers edition on x86 platform).

    Hope I am clear in mentioning my questions.

  4. Eden Xia says:

    it really address my issue . one of my sql server hung periodcally. after so long time investigation I reached this article. hope it can resolve my issue. trying now