Best Practices - SAN LUN Creation and Size Expansion for SAP Architectures

While dealing with customers over the last few months we have been encountering numerous scenarios of LUN Expansion using a methodology called Concatenation. This method has been causing some significant performance issues in SAP environments. Because of this it was decided that a blog needed to be created stating the proper methods for LUN size creation and expansion. We will first focus on the creation of LUNs using concatenation. As an example we will take a customer that we work with back in Q1 of 2008. The storage platform at the time was a XP 12000, which is currently at 25% utilization (bandwidth). The SAP ERP (ECC 6.0) system utilizes two ports on the XP and is connects to the SAN by way of a switch. A total of 10 LUNs were allocated for the data file volumes, one LUN for the Transaction Log and one for Tempdb. These LUNs were support by an underlying logical volume called a LuSE (LUN Size Expansion). A LuSE is a methodology used to create large LUNs or increase the size of a LUN being offered to the operating system. Each LuSE is comprised of at least two smaller devices called LDEVs (Logical Devices), however the customers system consisted of four LDEVs. In the screen shot below the turquoise container represents the Array Group. The green numbered components represents the LDEVs. Multiple LDEVs mean multiple slices that can be used for the same system or other systems (another blog on the do’s and don’ts of disk slice and disk sharing amongst applications later).

Each of the Array Groups which the LDEVs are carved from is in a RAID 5 (7D+1P) configuration for performance and fault tolerance.

The LDEVs were 56GB each in size and each carved from a different RAID Group (aka, Array Group). The method for carving an LDEV from the Array Group is referred to as horizontal slicing.

Based on best practices and experience with transaction based systems, a LuSE configuration are not recommended for high workload SAP environments. In a LuSE configuration an I/O operation is limited by the number of disks that can be used for the operation. In other words, if you have a LuSE comprised of e.g. two LDEVs from two different Array groups (RAID 5, 7D+1P) it seems you can only take advantage of the eight disks in the first LDEV or Array Group for read/write operation, since it is the only container written to in the beginning. This is due to the fact that a LuSE use concatenation to bring the LDEVs together. Concatenation of the LDEVs do not allow for writing across all spindles. For new systems this means all read/write operations will be on the first Array Group. As the first LDEV becomes full the current writes and reads will begin to occur on the second LDEV, with historical reads occurring on the first LDEV or Array Group. Hence the reasons why we state that concatenation of disk carvings from Array Groups are not a good practice for SAP ERP systems.

Generally, transaction intense applications like SAP ERP use a small random data block pattern to transfer data. With this type of data pattern, it is better to have more back-end drives available to service the I/O operation allows more host I/Os to be processed simultaneously and faster with a concatenated configuration this is not possible. So keep in mind that using LuSE or concatenation for increasing LUN sizes on HP XP12000, XP24000 and Hitachi Tagmastore (600 -> 1100) goes against best practices for building a high performance SAP system.

We have provided you with information regarding one of the problems we are encountering in the field with SAP systems, now what would be a better solution for LUN size expansion. You don’t want to walk in and tell the customer that using Concatenation or LuSE is a bad practice and not have an alternative. At least that is the architects game I play today. Your solution is striping either at the OS level or SAN level. Easy answer, use striping instead of concatenation for LUN size creation. How did I get to this conclusion? Well with the customer scenario we were involved in the SAP CoE team had to do some research for alternatives. While conducting the research we found documents that discussed Dynamic Disk/Veritas Volume Manager and SAN Level striping as a better alternative to using Concatenation or LuSE for LUN expansion. For Hitachi and HP systems with the available firmware a method called “disk interleaving is available for LUN creation. Disk interleaving the SAN to strip together 2 or 4 Array Groups (e.g. RAID 5, 7D+1P) so the I/O request can be serviced by more spindles (stripe across all spindles). This configuration can provide a significant performance improvement over a LUSE configuration. This information is based on best practices found in the HP StorageWorks XP12000 Disk Array Best Practices for Performance Guidelines whitepaper. More details on this in the next blog, next week...

All images used in this blog to explain SAN configurations come from HP whitepapers

Ciao