The msExchESEParamLogBuffers storage group attribute governs the number of Extensible Storage Engine (ESE) log buffers that are used by the Exchange information store. ESE uses a set of log buffers to hold information in RAM before it writes to the transaction logs. As you already know we have two sets of physical data in Exchange (physical data being physical files):
1. Transaction log files
2. Database Files (forget the .edb and .stm at the moment, just think of them as a single entity)
The exchange db file is made up of 4k pages structured across a given number of binary trees. Data is changed in the file by modifications to existing pages, while new data is added to the file through the creation of additional pages. Deletion follows the same process. From an operations point of view working with data through a physical file is not performance friendly. IO operations that have to reach physical disk (array cache) are costly performance wise for any application. Therefore the ESE engine (exchange's db storage engine) works with db page creations, modifications and deletions in volatile memory on the host (aka RAM). The db has a cache reserve in memory which is governed by the msExchESEParamCacheSizeMax attribute. Db pages are read from our physical file (disk/array cache) and placed into the reserve. This reserve is then used to work with pages - thus providing data modification performance. Changed pages (those that have been modified in some manor) are known as 'dirty-pages' whilst their unchanged cousins are referred to as 'clean-pages'. Dirty pages are eventually flushed from the db cache to disk - this happens during a variety of circumstances, that we don't need to go into here - just be aware that our dirty pages are eventually de-staged to physical file.
So, that's fine - that is the db - but where do the transaction logs come into it and especially the msExchESEParamLogBuffers attribute?
Exchange utilizes the Extensible Storage Engine (ESE),which is a transactional based database model that utilizes ACID. This is:
- Atomic: This term indicates that a transaction state change is all or none. Atomic state changes include database changes, and messages and actions on transducers.
- Consistent: This term indicates that a transaction is a correct transformation of the state. The actions taken as a group do not violate any one of the integrity constraints associated with the state. This requires that the transaction be a correct program.
- Isolated: This term indicates that even though transactions execute concurrently, it appears to each transaction (T) that others executed either before T or after T, but not both.
- Durable: This term indicates that once a transaction completes successfully (commits), its changes the state to survive failures.
Each of these 4 components are delivered by our transactional logging capability. Every change to a db page is considered a transaction, even though a complete transaction could (and 99.9% of the time does) involve the modification of more than a single page. The msExchESEParamLogBuffers attribute is exactly the same msExchESEParamCacheSizeMax attribute but it is for the transaction logs. As we change db pages to a dirty state in the db cache, we record the process (known as a transaction) to the physical log file. Just as working with data on physical disk for the db causes performance degradation, so it does for the transaction logs. Therefore we have a volatile memory reserve for our log operations too. The reserve size is governed by the msExchESEParamLogBuffers attribute.
Exchange's ability to recover from a crash comes in the form of a durable commit. A durable commit says once the transaction is logged into the msExchESEParamLogBuffers memory reserve, we must flush this to physical disk straight-way - prior to its corresponding dirty db pages being de-staged to the physical db file. This ensures we keep consistency for recoverability and the db never contains an incomplete transaction (aka corruption). The larger the msExchESEParamLogBuffers memory reserve, the greater the number of transactions that can be simultaneously worked on and the also the larger individual transactions can be.
So, that's what the msExchESEParamLogBuffers attribute is there for, increasing the size can provide better performance for multiple transaction recording operations. This is particularly beneficial to busy corporate Exchange severs. If the msExchESEParamLogBuffers becomes full we start to see log operation stalls, which directly affect the clients performance when changing data - that's why we say WRITE performance for transaction log volumes is critical - transaction records do not get flushed from memory until they have been through the durable commit.
The msExchESEParamLogBuffers attribute can also be taken into consideration during storage sub system testing. The objective of Jetstress is to generate an IO profile as much like a production Exchange server as possible. The msExchESEParamLogBuffers attribute is Active Directory (AD) based and therefore doesn't come into play with Jetstress. In a production Exchange environment the msExchESEParamLogBuffers attribute is read by the server and configured value is used for the memory reserve size. Referencing the above explanation in this mail for how transactional logging works, we can see that the msExchESEParamLogBuffers attribute can directly affect our IO write profile for the transaction log volume. For example: if the memory reserve size is small, we should see increased 'smaller' IO operations. The reason for this is because with a smaller buffer ESE is under a greater pressure to free up reserve space, ensuring we do not encounter a log stall scenario. On the other hand; if the memory reserve size is greater, we can expect to see reduced IO operations but of a larger size. The reasoning behind this is because ESE is not under so much pressure to flush every complete transaction immediately and therefore can group a number of complete transactions together for a single 'cost effective' write IO.
Jetstress has always had the capability for log buffer consideration (existing in the CMD line version and in the UI version). Up until recently, the UI version didn't support entering in a value greater than 8192 and by default was configured to use a value of 64. Jetstress now has the capability to adjust this value, and has a default setting of 9,000. Having a larger msExchESEParamLogBuffers size will prove advantageous for write performance to the log volume. The IO handoff is a single threaded, sequential process and we need to wait for a response back prior to issuing the next IO. Therefore, logic says - if we produce less IO, response times will be quicker as we will mitigate any risk of queuing IO operations on the host that are perhaps awaiting handoff to the storage device.
The new default value of 9,000 in Jetstress will provide advantageous to response time and generate less IO. This would be my recommendation. Providing the production Exchange Sever is configured in the same manor, the setting will produce the nearest profile to Exchange running under normal operations. However, there is a complication as always. Jetstress uses very small updates (replace of a single record). So, if you use 100% durable commits for its options then the size of the log buffer almost doesn't matter because we are always flushing it to commit the transactions to disk ASAP. If you use a mix of durable and lazy commits then this will take advantage of the larger buffer and will also more closely simulate the log behaviour of Exchange. Exchange has a lot of lazy commit transactions in a chain that are then followed up by a single durable commit to flush them all to disk. You should just view this as a larger single transaction because this is just an artefact of how Exchange is architected internally.
Frequently Asked Questions
[Question] Before tweaking log buffers should I check that the disks are behaving properly? i.e. If the disks are slow, will tweaking JET buffers really solve the root cause of the problem.
[Answer] If you are experiencing 'bigger' issues with the storage solution modifying the log buffer value will not make any difference. Although, one thing I would say is that this attribute is only a controlling factor for log IO. A value of 9,000 will purely provide a REAL simulation of Exchange IO. Remember the latest build of Jetstress to uses the 9,000 figure by default. So, whether you have 'poor' performing disks already, you should try to simulate Exchange IO as closely as possible - at least to base line your performance testing. Once you have a base line you can attempt to improve the performance of you storage solution following best practice guidelines from your chosen storage vendor and Microsoft.
[Question] When should we change the msExchESEParamLogBuffers attribute? when average Database\Log Record Stalls/sec exceeds 10ms, and spikes above 100ms?
[Answer] If an Exchange server is seeing log stalling behaviour then increasing the ESE log buffer should offer help. By increasing the value, and thus the memory reserve size we increase our ability to handle more transactions, thus not stalling them causing thread waiting behaviour. The Exchange performance whitepaper details the correct figures based on product team matrix testing. Another good reason for the increase (and why it should be done for a configuration step as part of a server install) is the larger the memory reserve size, the larger the transaction we can handle in single simultaneous process.
[Question] Will changing msExchESEParamLogBuffers to 9,000 increase or decrease the log disk write latency?
[Answer] Changing the log buffer value will change the log IO profile. Therefore, the answer is yes - it may well change the write latency figures. However, what I would say is this doesn't really matter a great deal. As I mention above the default value is now 9,000 - this is a 'recommended' performance tuning step which offers the best working environment for Exchange. Therefore we should work with this figure and perform other steps to ensure our log write latency is acceptable.
[Question] Does the recommendation of changing msExchESEParamLogBuffers to 9,000 apply to both E2K and E2K3?
[Answer] See the following article for the correct Exchange 2000 and Exchange 2003 configuration: