Memory Management – Dude where’s my RAM??

Only a couple of years ago, desktop machines with 4GB of RAM were rare.  Only high-end CAD designers and ultra-hardcore gaming enthusiasts were considering, let alone actually installing, 4GB of RAM in their machines.  Today however, RAM is much less expensive, and system builders such as Dell or HP routinely offer the option to build  a system with 4GB of RAM.

Because of this, we are often asked why Windows Vista / Windows Server 2003 / Windows XP etc. only sees a portion of the installed memory.  It is pretty common for a machine with 4GB of RAM to actually show between 3.1 and 3.6GB of RAM installed, although this number may actually be less depending on hardware.  The screenshot below is from a Vista Ultimate RTM machine with 4GB of RAM installed – notice how much RAM is reported as being installed:

It is important to understand that this is not really a problem with the OS, but is more hardware related.  Let’s talk about the OS first since it is the simpler part of the issue.  On any 32-bit Operating System (not only Windows), you only have access to 4GB of address space by default.

We discussed this in a previous blog post: Memory Management 101.  A 32-bit Operating System can actually handle 4GB of memory.  Using the /PAE switch allows the OS to handle memory above this range as long as the application is designed for it.  Keep this in mind because it will come up again after the next section: hardware.

The issue is the way in which the hardware allocates memory for its own resources.  The hardware needs to allocate memory space to use for things like the PCI bus, BIOS, the video card and others.  It allocates this from the address space presented to it, which is not necessarily the same as the amount of physical RAM installed. Also of note, it allocates this memory from top to bottom.  The problem is, when you have 4GB of RAM installed, the amount of physical memory installed is the same as the address space.  I think you can see where this is going.

If you have 4GB RAM, and the hardware needs to allocate a large chunk of memory for its own use, and it does this from top to bottom, the memory that is blocked off starts at 4GB and allocates downwards.  So, the final amount of RAM the OS will be able to see is the difference.  This is because when it actually allocates for the physical RAM in the system, it has to skip the chunk that was blocked off by the hardware.  Since a 32-bit OS can only see 4GB, the rest of the RAM is invisible because it is above the 4GB barrier.  By using the /PAE switch, you enable the OS to see above this barrier, and you can see all of your RAM.  Sometimes.

The real problem comes back to hardware.  As I am sure you can imagine, the OS can only handle whatever resources are shown to it by the hardware BIOS.  If the hardware does not support a large enough addressing range, then it simply won’t report anything above that so the OS is in the dark.  If the hardware supports 36-bit PAE Intel Extensions or the AMD equivalent, and you use an OS that supports PAE, you should be able to enable both and see all of the RAM.

The resources below go into additional depth regarding this particular topic.

Additional Resources:

 – Tim Newton