“FSCT” - A CIFS/SMB/SMB2 File Server Tool for Capacity Planning and Performance Troubleshooting

Overview

A recent presentation during SNIA’s Storage Developer Conference described an upcoming tool called “FSCT”.
“FSCT” (which stands for “File Server Capacity Tool”) was created by Microsoft after analyzing the workload of production file servers used to store user’s home folders.
The tool can simulate that typical file server workload (and potentially others) in a lab environment.

What is “FSCT”?

“FSCT” is a tool for CIFS/SMB/SMB2 File Servers focused on capacity planning.  It’s also useful for identifying bottlenecks.
“FSCT” can simulate multiple TCP connections and sessions, with each client test machine simulating multiple users.
It can also exercise certain sequences of file operations to simulate real applications’ behavior.
Results provided by the tool include the maximum number of users for a server configuration, throughput for a server configuration (measured in scenarios per second), scenario response time and performance counters for servers and clients.
The target audience for “FSCT” comprises IT Professionals and Storage Solution Providers.

“FSCT” History

“FSCT” was originally created as an internal tool developed by the Windows Server Performance Team.
It is used internally every day in Windows Server Labs, where the team focuses on server quality gates and regression testing.
In short, the tool checks the daily builds of Windows to compare the file server performance of the latest daily builds against previous ones.
To make this internal tool externally available, the team is simplifying the command line interface, updating the workloads and creating additional documentation.

“FSCT” Configurations

In order to run “FSCT”, you need at least three computers.
The first one, called the Server, is the system under tests. The Server can run any OS that supports CIFS, SMB or SMB2.
The second one, called the Client, runs the workloads against the Server, simulating multiple users.
The third one, called the Controller, is responsible for coordinating the work of the Clients, capturing performance counters and compiling the results.
With “FSCT”, you can have many Clients, but only one Server and one Controller.

A sample configuration could include, for instance, one dual core server, 8 single-core clients and a single-core controller.
The server could include 6 volumes on 12 spindles and 4 network Gigabit Ethernet cards.
There are no hard limits in the tool itself, but we have tested it with a server with 16 cores, 32GB of RAM, 100 spindles on a disk array, 5 NICs and 120 clients.

Workloads and Scenarios

One important “FSCT” feature is the ability to use workloads and scenarios.
A scenario is a basic user action being simulated, like deleting a file from the command line, copying a file with Windows Explorer or opening a file in Microsoft Word.
A file set is required by each scenario.
A workload is a set of well-defined scenarios and file sets.

The workload definition is provided in the form of XML files, including the parameters required for each scenario, like the frequency in which they should occur and the file set used.
The actual code simulating a scenario is provided in the form of DLL files.
Only one workload, called HomeFolders, is planned for the initial release of “FSCT”.

The HomeFolders Workload

The HomeFolders workload was created by capturing data from real, heavily accessed file servers and their clients.
Data captured included Event Tracing for Windows (ETW) traces on servers, client API traces and network traces.
The Windows Server Performance Team worked with Microsoft IT administrators to enable tracing for a few minutes each hour over many months.

The data was then analyzed, looking at request types distribution, parameters distribution, file set characteristics and connection characteristics.
Finally, a developer created the workload, testing to verify it achieved approximately the key findings from the traces.
The workload development effort included creating the scenarios, creating the file sets and defining the number of runs per user per hour of each scenario.

It is important to mention that “FSCT” and the HomeFolders workload were developed independently.
The “FSCT” architecture allows for the creation of custom workloads and scenarios, as well as tweaking the existing workloads.
However, as mentioned previously, only the HomeFolders will be included in the initial release of “FSCT”.

Running “FSCT”

Running the tool includes three distinct phases: prepare, run and cleanup.
During the prepare phase, a command line must be run on the server, client and controller. This will initialize volumes, create users, update the hosts file and update configuration files.
During the run phase, a command line must be run on each client and on the controller.
The controller will initiate a series of runs, capturing performance counters and results, all of which are consolidated into a set of files in the controller.
Finally, the cleanup phase will remove all users, files and configuration settings put in place during preparation.

“FSCT” White Paper

Along with the tool, Microsoft plans to provide a white paper containing an overview of CIFS/SMB/SMB2 and an overview of “FSCT”.
The white paper will also include step-by-step instructions on how to use the tool with both Windows and non-Windows servers.
It will also include a command-line reference, annotated results file and a description of the HomeFolders workload.

Planned Release

There is no public version of “FSCT” available at this time.
The initial release of “FSCT”, along with the white paper, is planned for 2009.