Selecting the number of nodes for your Scale-Out File Server Cluster

I recently got a stream of e-mails and questions about the maximum number of cluster nodes you can have in a Scale-Out File Server cluster. For the record, we test and support up to 8 nodes per file server cluster. This is the case for both Windows Server 2012 (which introduced the Scale-Out File Server cluster feature) and Windows Server 2012 R2.




However, the real question is usually “How many file server nodes do I need for my Scale-Out File Server cluster?” The most common scenarios we see involve the deployment of 2 to 4 file server nodes per cluster, with just a few people considering 8 nodes. Here are some arguments for each cluster size:

  • 2 is the bare minimum for achieving continuous availability (transparent failover). We see a lot of these two-node clusters out there.
  • 3 is a good idea to allow you to still have continuous availability even when you’re doing maintenance on a node.
  • 4 allows you to upgrade the cluster without extra hardware by evicting two nodes, installing the new operating system and using the Copy Cluster Roles Wizard.
  • 8 will allow you to combine the network throughput and computing power of the many nodes to create an amazing file sharing infrastructure.

On the topic of performance, keep in mind that the Scale-Out concept means that you have the ability to linearly scale the cluster by adding more nodes to achieve higher IOPS and throughput. We have proved it in our test labs. However, we have also shown that a single file server cluster node can deliver over 200,000 IOPs at 8KB each or over 2GB/sec throughput, with a fairly standard server configuration (two modern Intel CPUs with a few cores each, dual 10GbE RDMA network interfaces, two SAS port each with four lanes of 6Gbps SAS shared storage with an SSD tier). We have also shown high-end file server configurations achieving over a million IOPs and over 16.5GB/sec from a single node. So, there are only a few scenarios that would require anything close to 8 nodes solely to achieve specific performance goals.

Also, keep in mind that the reasons for adopting a Scale-Out File Server usually include a combination of availability, performance and scalability of the resulting file service. While the number of nodes is an important ingredient to achieve that, you should never overlook related configuration decisions like the number and model of CPU you’re using, the number and type of network interfaces, the number and speed of the SAS ports, the proper deployment of tiered storage with the right class of SSDs and even the generation and number of lanes of the PCIe slots you use. Two well sized and properly fitted file server nodes can easily beat eight nodes that were poorly put together.

Here a few additional references for those looking to dig deeper:

I’m not sure what’s driving this wave of questions about the Scale-Out File Server cluster limits, but I hope this helps clarify the topic. Have you ever encountered a scenario where you would need more than 8 nodes? We would love to hear it…

Comments (22)

  1. ML49448 says:

    Another important item to consider with SAS storage is whether the server supports PCI express 3.0

    you need PCIe 3.0 x8 to support the bandwidth a dual port SAS card like the LSI 9207-8e can generate

  2. JoseBarreto says:


    PCI Gen3 x8 goes up to about ~6.7 GB/sec, which can keep up with 2 ports of 6Gbps/4-lane SAS (total of 4.4 GB/sec from both ports combined). If using PCIe Gen2 x8 or PCIe Gen3 x4, the bus becomes a bottleneck.

    See tables at the end of…/hyper-v-over-smb-performance-considerations.aspx for further details on speeds and feeds…

  3. Andre Muraro says:

    Hey Jose,

    Great article!!

    From my point of view the amount of RAM is another important decision for each node since you can setup up to 80% of RAM exclusively for cache on CSV volume.

    A question for you:  I guess is strongly recommended to have all nodes with the same hardware configuration to avoid unbalanced configuration, and surprises during fail-over process, am I right?

  4. JoseBarreto says:


    Yes, RAM for CSV caching is important and balanced nodes is always a good idea to make things more predictable.

    To be fair, whatever passes Failover Cluster Validation is supported, but I would keep them configured exactly the same, like monozygotic twins.

  5. Sharbel says:

    Dear Jose,

    I have a question out of the above scope please.

    You mentioned that always we need to choose the smaller number of the two tiers SSD+HDD for the NumberOfColumns.

    The question if I added more disks in the JBOD to expand the storage space in the future.

    The example we have 4 disks in the SSD tier and 8 disks in the HDD tier for a mirrored space.

    I will add 4 SSDs and 4 HDDs (Total= 8 disks in the SSD tier and 12 disks in the HDD tier).

    How this will reflect the NumberOfColumns?

    Before we expand the JBOD, the NumberOfColumns was set to 2 for the mirrored space (4 SSD/2).

    Do I need to run a new cmdlet below and modify the NumberOfColumns from 2 to 4?

    Get-StoragePool Pool1 | Set-ResiliencySetting -Name Mirror -NumberOfColumnsDefault 4

    I am setting Storage Tiering on a new HP server with 25 Bays mixed between HDD and SATA SSD.

    I noticed that the Storage Pool detect the media type for the SSD disks as SSD, but for the HDD is showing as UNKNOWN.

    Is it safe to SET the PhysicalDisk -MediaType = HDD in production and not for lab test?

    Is it recommended that the Server must detect the media type of the disks by default or we can set it in PowerShell if it showing Unknown?

    Thank you in advance.


  6. Yoshihiro Kawabata says:

    Can I use Single Share for Scale-Out File Server with minimam Redirection of CSV Data/Metadata at Mirrored Storage Space ?

    In your article "Automatic SMB Scale-Out Rebalancing in Windows Server 2012 R2",

    at section "Hyper-V over SMB with Windows Server 2012 R2 and Mirrored Storage Spaces (asymmetric)"

    " In this new release,

     the SMB client gained the flexibility to connect to different Scale-Out File Server cluster nodes

     for each independent share that it needs to access"

    AND, at section "Key configuration requirements for asymmetric storage in Windows Server 2012 R2"

    " To be clear,

     if you have a single CSV in such asymmetric configuration

     in Windows Server 2012 R2 Scale-Out File Server cluster,

     only one node will be actively accessed by SMB clients"

    I want to use Mirrored Stoage Space, and 4 Hyper-V Cluster node, 10 Guest OS per node,

    How many Share do I need ?


    I like Windows Server 2012 R2 Storage Space

    Yoshihiro Kawabata

  7. JoseBarreto says:


    If you use a two-node Scale-Out File Server with a single share backed by a mirrored storage space in Windows Server 2012 R2, all clients (Hyper-V hosts) running Windows Server 2012 R2 will be redirected to a single file server node, the owner node for that single CSV. The other file server node will be idle waiting for a potential failover.

    This is not about the number of Hyper-V nodes, but the number of file server nodes in the configuration. You mentioned you had 4 Hyper-V nodes. Assuming you have 2 file server cluster nodes, you should use at least 2 shares, each backed by a different CSV on a mirrored storage space. Each of the 4 Hyper-V nodes will connect to the file server node that owns the CSV for that share. If a single Hyper-V node is using both shares, that node will connect to both file server nodes and use the best path for each share.

  8. JoseBarreto says:


    It’s safe to set the media type property if it’s not automatically detected. Just make sure you  set it to the correct value (HDD or SSD) and don’t place any data on it before doing it.

    You can’t change the number of columns for an existing space (the number of columns is a property of the storage space, not the storage pool). You can add disks to the pool to get more capacity and you can increase the size of the space (using Resize-VirtualDisk),
    even if it’s tiered (use Resize-StorageTier). However, the number of columns has to remain the same, since it dictates how data is laid out on the disk.

    You can find the Storage Spaces FAQ at

  9. Ricky Day says:

    What a brilliant post, much appreciated Jose a tonne of information there 🙂

  10. Yoshihiro Kawabata says:

    @Jose Barreto — Thank you for clear response.

    I will continue to try Windows Server 2012 R2 Scale Out File Server.


  11. dgrambo says:

    What you're doing with  SMB is refreshing. I look forward to seeing the move towards leveraging the physical desktop and I hope youre involved.

  12. Sharbel says:

    Thank you dear Jose for the clear response,

    I am having an issue to (Resize-StorageTier) and expand the SSD_Tier:

    I still have 7GB free on the SSD Tier:

    PS C:Get-StorageTierSupportedSize MirroredTieredSpace1_SSD_Tier -ResiliencySettingName Mirror | FT -AutoSize

    SupportedSizes TierSizeMin TierSizeMax TierSizeDivisor

    ————–        ———–   ———– —————

    {}                    1073741824  7516192768      1073741824

    When I try to expand the SSD tier using below cmdlet:

    PS C:Resize-StorageTier -InputObject (Get-StorageTier -FriendlyName "MirroredTieredSpace1_SSD_Tier") -Size 3GB

    Resize-StorageTier : Invalid Parameter

    At line:1 char:1

    + Resize-StorageTier -InputObject (Get-StorageTier -FriendlyName "MirroredTieredSp …

    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

       + CategoryInfo          : InvalidArgument: (StorageWMI:ROOT/Microsoft/…SFT_StorageTier) [Resize-StorageTier], CimException

       + FullyQualifiedErrorId : StorageWMI 5,Resize-StorageTier

    PS C:Get-StorageTier *Space1*

    ObjectId             : {1}\ISCKC-HV01root/Microsoft/Windows/Storage/Providers_v2SPACES_StorageTier.ObjectId="{6fa45e22-498a-11e3-80b4-806e6f6e6963}:ST:{5c3a3723-4ab2-11e3-80bb-38eaa78c3d44}{5c3a37ae-4ab2-11e3-80bb-38eaa78c3d44}"

    PassThroughClass     :

    PassThroughIds       :

    PassThroughNamespace :

    PassThroughServer    :

    UniqueId             : {5c3a37ae-4ab2-11e3-80bb-38eaa78c3d44}

    Description          :

    FriendlyName         : MirroredTieredSpace1_SSD_Tier

    MediaType            : SSD

    Size                 : 375809638400

    PSComputerName       :

    ObjectId             : {1}\ISCKC-HV01root/Microsoft/Windows/Storage/Providers_v2SPACES_StorageTier.ObjectId="{6fa45e22-498a-11e3-80b4-806e6f6e6963}:ST:{5c3a3723-4ab2-11e3-80bb-38eaa78c3d44}{5c3a37af-4ab2-11e3-80bb-38eaa78c3d44}"

    PassThroughClass     :

    PassThroughIds       :

    PassThroughNamespace :

    PassThroughServer    :

    UniqueId             : {5c3a37af-4ab2-11e3-80bb-38eaa78c3d44}

    Description          :

    FriendlyName         : MirroredTieredSpace1_HDD_Tier

    MediaType            : HDD

    Size                 : 2394444267520

    PSComputerName       :

    Thank you in advance.


  13. JoseBarreto says:


    On the Resize cmdlet, you have to specify the new total size, not the amount you want to increase.

    You can find an example of how to use Resize-StorageTier (and the required Update-Disk) in item 22 of this blog post:…/step-by-step-for-storage-spaces-tiering-in-windows-server-2012-r2.aspx

  14. Günter says:

    Hey Jose,

    we want to use 2 Deployment units ( as shown in TechNet Wiki Storage Spaces Overview) with 2 nodes and 1 JBOD encosure each. Now my question: can I pool disc from both deployment units ?

    And if not: hoe can i mirror the data between the units?


    Günter Herrwaldt

  15. JoseBarreto says:


    You can have a mirror with disks on the same JBOD.

    To mirror across 2 different JBODs, they must be connected to the same hosts.

    If there are different hosts connected to different JBODs, you can use Hyper-V Replica.

  16. Sharbel says:

    Thank you dear Jose for your prompt reply.

    I have a Server with 24 bays (18 HDDs + 6 SSDs). What is the best practices to create a Storage Space with tiering (Mirror) to store Virtual Machines? One virtual disk as one volume for all the 24 disks or two virtual disks 2 separate (Mirror each volume) for better performance?


  17. JoseBarreto says:


    You can probably do a single pool and a few virtual disks.

    You generally should keep your virtual disks smaller than 64TB (there is a limit related to snapshots). If you're using large disks (4TB) and 2-way mirroring, you'll likely have 48TB of usable disk space, so a single virtual disk is a possibility.

    However, in a Windows Server 2012 R2 Scale-Out File Server cluster, you want to have at least as many virtual disks as you have nodes. So, for a two-node cluster and the same disk sizes, you could use two virtual disks using 24TB each.

  18. Rich Stephenson says:

    Great Info! I'm building this Scale out File Server scenario now in our lab. A general question. I have three network cards per SoFS node: a dual RDMA (iWARP) card and two 1GbE cards. Should the RDMA nics have IPs on my CorpNet lan? Or should my RDMA nics each have dedicated networks?

    I'm guessing I also want a private heartbeat network as well. Pretty cool architecting this stuff.

  19. JoseBarreto says:


    You can connect the RDMA network to your corporate network or you can keep it separate. Your choice. Just make sure you have a redundant path to your DNS server, so the client can find the file server. From there, the client and server can find the RDMA path even if that is a private network.

    You also do not need a dedicated private heartbeat network. You can use either the 1GbE cards or the RDMA NICs for that.

    You might want to review this post:…/networking-configurations-for-hyper-v-over-smb-in-windows-server-2012-and-windows-server-2012-r2.aspx

  20. Rich Stephenson says:

    Thanks Jose. Follow up question. So if both my RDMA nics are on my corporate network, then I have only a single network listed under Failover Cluster Networks? Would that be correct? So for my storage cluster, all of my nics would be on my corporate network (except iSCSI networks)?

  21. Anonymous says:

    In this post, I'm providing a reference to the most relevant content related to Windows Server 2012

  22. Anonymous says:

    In this post, I’m providing a reference to the most relevant content related to Windows Server 2012