Exchange 2003 Memory Configuration change for Windows 2003 (PAE Support)


Introduction


Exchange 2003 requires specific memory tuning to function optimally.  This flash outlines Windows 2003 boot.ini configuration settings for Exchange 2003 and highlights recent changes.  See the following KB articles for relevant background information on this topic:


·         CPU and Memory Scalability for Exchange Server 2003 and for Exchange 2000 Server


·         Use of the /3GB switch in Exchange Server 2003 on a Windows Server 2003-based system


·         Using the /USERVA switch on Windows Server 2003-based computers that are running Exchange Server


 


This Flash covers only Exchange 2003/Windows 2003 topologies.


Deployment Guidelines


Exchange 2003 requires Windows 2003 Sp1 or Windows 2003 with security update MS04-032.  Both of these versions contain the fix for a problem described in article 834628 which is necessary for Exchange 2003 to run reliably.  The following table outlines the boot.ini configuration switches that must be added for the given Exchange 2003/Windows 2003 server role:


 


Boot.ini Configuration Matrix:  Exchange 2003 RTM/SP1+ , Windows 2003 RTM/SP1+


 
















































Exchange 2003 Server Role


Physical Memory Config


Additions made to Boot.ini


Mailbox


> 1 GB**


/3GB /USERVA=3030


 


 


 


Public Folder


> 1 GB**


/3GB /USERVA=3030


 


 


 


Front End (FE)


> 1 GB**


None


 


 


 


SMTP Gateway/Bridgehead


> 1 GB**


None


SMTP Gateway/Bridgehead (Envelope Journaling*)


> 1 GB**


/3GB /USERVA=3030


 


 


 


MTA/X.400/3rd Party Connector Bridgehead*


> 1 GB**


/3GB /USERVA=3030


 


* Envelope Journaling or MTA/3rd Party Connectors send Gateway/Bridgehead email through the Store.exe process so the system benefits from /3GB.


** /3GB and /USERVA=3030 is not used on Exchange servers with less than 1GB RAM


***See KB 823440 for additional details on configuring Windows memory for Exchange 2003.


****Windows may add/require additional boot.ini configuration switches.


*****The above settings are applicable to all Exchange 2003 and Windows 2003 server SKU’s including Standard Editions.


Support Policy/Recommendation Change


There has been significant change to Exchange 2003’s recommended Boot.ini configuration settings.  The following change has been made to Microsoft Exchange 2003 Support Policy.


 


It is recommended and supported to run Exchange 2003 with the PAE (Physical Address Extension) kernel on Windows 2003:  Due to the issue (Hotfix in article 834628)  found in Windows 2003 RTM running the PAE kernel, the fact that Exchange does not take advantage of more than 4GB of memory and the fact that Exchange running on PAE kernels was not initially tested;  the Exchange Product group originally recommended running the /NOPAE switch in the boot.ini to prevent  the PAE kernel from loading.  This decision has been reversed for the following reasons:


 


a.       Windows 2003 SP1 was released:   


·         Windows 2003 Sp1 contains the fix for 834628 so Exchange 2003 is stable running on the PAE kernel.


·         Windows 2003 SP1 contains new security features (Data Execution Prevention (DEP)) which require the PAE kernel.  Exchange 2003 needs to support PAE to take advantage of these new security features.  Windows 2003 SP1 will automatically set DEP specific boot.ini settings to take advantage of DEP capable hardware.


b.      New PCI-Express based server chipsets require PAE to take advantage of all the memory installed on a system (4GB):


Some of the physical address space on server systems is used to provide memory mapping of IO resources on the system chipset. This memory mapped IO (MMIO) space is typically provided below the 4GB address boundary. In a system with 4GB of physical memory this MMIO space pushes a section of physical memory above the 4GB address boundary. If the software running on the server supports only 32-bit physical addressing, it will not look for memory over the 4GB boundary. This results in the OS not providing access to all 4GB of physical memory. The amount of ‘hidden’ memory is equal to the address space taken up by the MMIO.  PCI Express confounds this problem by providing extended PCI configuration space to support such features as Advanced Error Reporting (AER). This means the size of MMIO space required to map the chipset IO resources gets larger. Some early PCI Express chipsets confound the issue further by providing coarse granularity for this mapping which results in a range of 512MB being carved out for MMIO. This results in customers reporting hidden memory of 512MB to 768MB on servers using such chipsets.


c.    The Exchange Product Group tested Exchange 2003 SP1/SP2 on Windows 2003 (/w 834628 ) with the PAE kernel and on Windows 2003 SP1 with PAE and DEP enabled:


·         The Exchange Product Group did not find any issues relating to running on the PAE kernel nor running with DEP enabled (software and hardware enabled).



Additional Resources/Questions





  1. How can the boot.ini configuration be validated in a running environment?  Use the  Exchange Best Practices Analyzer Tool  The tool has been updated to include the support policy change outlined in this Flash.


  2. How should Exchange 2003 memory based performance issues be troubleshooted?  Use the Troubleshooting Exchange Server 2003 Performance whitepaper.


  3. What is Microsoft’s stance on running Exchange on Windows 2000 with PAE?  It is supported but not recommended.  Hotfix 838647 is required for support.  Windows 2000 does not have the security features that require PAE so the only benefit of running PAE on Windows 2000 for Exchange is to enable all memory access on newer PCI-Express based servers.  It is not a well tested scenario so it is not recommended.


  4. Does Exchange utilize more than 4GB of memory if the PAE kernel is loaded?  Exchange will only utilize 4GB of memory regardless of whether the PAE kernel is loaded or not.  The PAE kernel is beneficial (specifically for Exchange) to enable DEP and to see 100% of the physical memory on newer server chipsets.  Memory above 4GB will not be utilized by Exchange.

The Exchange Team

Comments (12)
  1. Anonymous says:

    Exchange Server Team blogs about the policy/recomendation change in relation to the use of the /NOPAE switch in servers running both Windows 2003 and Exchange 2003

  2. Susan says:

    Can you guys comment once and for all as to whether we should use the /3 switch in SBSland?

    We say no.

    Some say yes.

  3. Teo Heras says:

    Excellent information! We had to add the /nopae switch to all of our boot.ini files because JetStress was failing on servers with hot-swapable memory boards.

    Thanks!

    Teo

  4. Ben Winzenz says:

    Paul Bowden indicated to me at Teched this year that if you were using PAE, you should change the USERVA= option to 2970 instead of 3030. Is this no longer correct? Is it now recommended to leave it at 3030?

    Ben

  5. Jeff says:

    This article isn’t clear as to when to use the PAE switch. Is it only if your server has 4GB or more of physical RAM?

  6. Jeff says:

    <tpa><tap><tap> Is this thing on?

    With this new recommendation, when is it appropriate/recommended to use the /PAE switch? How much RAM should you have to use it?

  7. Frank says:

    Ok, so if you use the PAE switch, do you still also use the /3GB switch?

    And is there a verdict on the USERVA= value?

  8. Scott says:

    What about those of us in SBS land? To do or not to do….

  9. Ben Winzenz says:

    Jeff,

    PAE is automatically enabled in at least 2 instances. 1 – if a hot-add memory device is installed; 2 – if Data Execution Protection (DEP) is enabled via a chipset that supports DEP, or software DEP.

    http://support.microsoft.com/default.aspx?scid=kb;en-us;283037

    and
    http://support.microsoft.com/default.aspx?scid=kb;en-us;900524

    both confirm this.
    http://support.microsoft.com/default.aspx?scid=kb;en-us;827281 indicates the registry key you can check to see if PAE is automatically enabled. If set
    to 1 (technically, 00000001), the PAE kernel will automatically load at startup.

    Ben
    Exchange MVP

  10. Kevinm says:

    Ben,

    3030 is the standard recommendation. You then monitor the Memory Free System Page Table Entries perfmon counter when the server is loaded. If the value reported by perfmon is <8000, then change Boot.ini to 2970.

    Kevinm

  11. Korbyn says:

    Item #1, basicially we don’t need to worry about using /PAE or /NOPAE anymore, if the motherboard can use it, the system turns it on, otherwise, just leave it alone.

    Item #2, I did find an issue and reported it to MS that Windows 2003 SP1 with DEP/PAE enabled on an Cluster, the Exchange SMTP services didn’t function correctly. On one node the server wouldn’t start, on the other node the service couldn’t stop. Once I replaced the /noexecute switch with /Execute, service returned to normal. /Execute btw, is the only way to shut off both PAE and DEP.

    Item #3, I found changing the userva to 2970 didn’t improve my free PTE’s any on my clustered Exchange servers which are varying between 3800 and 5100. and these boxes are tuned to the max, and am wondering if PAE would help out. They all currently have the /NOPAE switch in, and we haven’t gone to production with SP1 for Windows 2003.

  12. Anonymous says:

    http://support.microsoft.com/kb/283037 My 64bit Windows 2003 (Standard) systems don’t need this but even good ole Windows 2003 Enterprise won’t pay any attention to more than 4GB of memory unless the /PAE switch is on. If you have Exchange, you will probably..

Comments are closed.