The Basics of Single Instance Storage (SIS) in WSS 2003 R2 and WUDSS 2003

  • Article

What is SIS

Single Instance Storage (SIS) is a feature of Windows Storage Server (WSS) 2003 R2 and Windows Unified Storage Server (WUDSS) 2003 that helps save disk space by de-duplicating files.

If you have two or more identical files in a volume, SIS will store a single copy of those bits, replacing the files with links (or “reparse points”) to that single store.

The files don’t need to be on the same folder, have the same name or have the same date, but they do need to be in the same volume, have exactly the same size and the contents of both need to be exactly the same.

As you enable SIS on each volume, a “SIS Common Store” folder will be created on that volume. This is where the single copy is stored (as a file with the SIS extension).

A service (called Groveler) will monitor the volume enabled for SIS to find and de-duplicate identical files.

The History of SIS

SIS was originally introduced as part of the RIS (Remote Installation Services) in Windows Server 2000, but at that time it was restricted to volumes used by RIS to hold multiple installations of Windows.

RIS has since been replaced by Windows Deployment Services (WDS), which leverages the WIM file format for the de-duplication of Windows installation files.

Today, SIS is only included as part of WSS 2003 R2 and WUDSS 2003.

SIS is not available in Windows XP, Windows Server 2003, Windows Server 2003 R2, Windows Vista or Windows Server 2008.

Checking if SIS is enabled

You can check if SIS is enabled on a volume by looking at the “Advanced” tab for the properties of that volume.

To get there, Click “Start” then “My Computer” and on the list of volumes right-click the disk and select “Properties”. Then go to the “Advanced” tab.

You should see something similar to one the windows below:

SIS 1
 
In the left you have a system without SIS enabled. On the right SIS is enabled on the system and the checked box indicates it’s also enabled for that specific volume.

You can also check if SIS is enabled using a command-line. You will use the SISADMIN.EXE tool for that. If you just run SISADMIN with no parameters, you will get:

 C:\> sisadmin

Valid commands:
    /? /h           Displays usage information
    /i     Install SIS (volumes must be specified)
    /f     Set the groveler to foreground mode
    /b     Set the groveler to background mode
    /r     Initiate a full scan
    /e              Enable and start the groveler service
    /d              Stop and disable the groveler service
    /v     Analyze volumes for overall SIS usage information
    /l     List SIS controlled files
    /s       Display information about SIS files
Where:
           Is a list of volumes of the form "e: f: g: q:". If no volumes are specified, the operation will be performed on all volumes where SIS is enabled.
             Is a list of space delimited files.

SIS is not enabled on any volumes.

C:\>

Note the message on the SISADMIN output indicating that SIS is not enabled.

You can also check the status of the Groveler service. It is initially stopped and disabled.

 C:\> sc qc Groveler

[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 4   DISABLED
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\WINDOWS\system32\grovel.exe
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Single Instance Storage Groveler
        DEPENDENCIES       : SIS
        SERVICE_START_NAME : LocalSystem

C:\>sc query Groveler

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN))

        WIN32_EXIT_CODE    : 1077  (0x435)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

C:\>

Enabling SIS

Before you enable SIS, you need to make sure the components are added to your system. This is done by default on WSS 2003 R2 and WUDSS 2003, but it does not hurt to check.

SIS should be in the component list you can access from “Control Panel”, “Add/Remove Programs”, “Add/Remove Windows Components”.

From there, select “Other Network File and Print Services”, click on “Details” and confirm that “Single Instance Storage” is checked like in the window below:

SIS 2

The next step is to enable SIS on the system. There is no GUI for that, so you will need to use the /i option of the SISADMIN command line tool, making sure to specify the volume (or volumes) where you want to install SIS.

There are a few restrictions. The volume must be formatted with the NTFS file system, it has to be a fixed disk and you cannot do it on the volume where Windows itself is installed. There is also a limit of 6 volumes per server.

Here is what you should see while enabling SIS on the E: disk:

 C:\>sisadmin /i E:

SIS has been initialized on volume 'E:'.
Restarting the groveler service to begin groveling new SIS volumes.

C:\>sc qc Groveler

[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\WINDOWS\system32\grovel.exe
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Single Instance Storage Groveler
        DEPENDENCIES       : SIS
        SERVICE_START_NAME : LocalSystem

C:\>sc query Groveler

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0


C:\>sisadmin /v

Analyzing volume 'E:\'.
Analyzing the SIS Common Store directory...
Analyzing the SIS Common Store directory... 0 processed, complete.
Sorting SIS common store file list...
Sorting SIS common store file list... complete.
Analyzing reparse points...
Analyzing reparse points... 0 processed, 0 not analyzed, complete.
Analyzing data...
Analyzing data... complete.
=== Analysis of volume 'E:\' on Server1 ===
Common store files:      0
Link files:              0
Inaccessible link files: 0
Space saved:             0 KB

C:\>

You can see that the “Groveler” service is now running and set to auto-start.

Testing SIS

Now we move on to actually prove that SIS is working, by creating a few duplicate files and verifying if the Groveler does its work.

At this point, all you need to do is add a few files that are copies of each other, wait for the Groveler to act and verify the space savings from SIS.

Here’s an example:

 E:\>copy C:\WINDOWS\system32\shell32.dll E:\File1.DLL

        1 file(s) copied.

E:\>copy C:\WINDOWS\system32\shell32.dll E:\File2.DLL

        1 file(s) copied.

E:\>copy C:\WINDOWS\system32\shell32.dll E:\File3.DLL

        1 file(s) copied.

E:\>dir

 Volume in drive E is New Volume
 Volume Serial Number is E870-DC26

 Directory of E:\

11/07/2007  05:49 AM         8,360,448 File1.DLL
11/07/2007  05:49 AM         8,360,448 File2.DLL
11/07/2007  05:49 AM         8,360,448 File3.DLL
               3 File(s)     25,081,344 bytes
               0 Dir(s)  17,077,125,120 bytes free

E:\>sisadmin /v

Analyzing volume 'E:\'.
Analyzing the SIS Common Store directory...
Analyzing the SIS Common Store directory... 1 processed, complete.
Sorting SIS common store file list...
Sorting SIS common store file list... complete.
Analyzing reparse points...
Analyzing reparse points... 3 processed, 0 not analyzed, complete.
Analyzing data...
Analyzing data... complete.
=== Analysis of volume 'E:\' on Server1 ===
Common store files:      1
Link files:              3
Inaccessible link files: 0
Space saved:             16336 KB

E:\>

You can also tell that a file has been de-duplicated by SIS by looking at its properties in Windows Explorer:

SIS 3 

You can see that the size of the file does not change, but the “Size on disk” property shows only 4KB.

SISADMIN also has a command line to tell exactly which files have been moved to the “SIS Common Store” in more detail.

Here’s that output:

 E:\>sisadmin /l E:

Listing SIS controlled files on volume 'E:'.
EE3C0955-AF46-11DC-BBF5-0003FF6F7322.sis <- E:\File1.DLL
EE3C0955-AF46-11DC-BBF5-0003FF6F7322.sis <- E:\File2.DLL
EE3C0955-AF46-11DC-BBF5-0003FF6F7322.sis <- E:\File3.DLL
3 SIS controlled files found on volume 'E:'.

E:\>

The SIS Common Store

In case you’re wondering, the “SIS Common Store” folder has special permissions to prevent users from seeing it or changing it in any away.

What’s inside is just a set of configuration files, the database used by SIS for tracking changes to the volume and the SIS files themselves.

Here’s what it looks like:

SIS 4

You should not change the permissions of the “SIS Common Store” folder or attempt to modify those files directly.

Always use the SISADMIN tool to manage SIS.

Disabling SIS

You can use SISADMIN also to disable and stop the Groveler service from running. Here’s how you would do it:

 E:\>sisadmin /d

Stopping and disabling the groveler service.

E:\>sc query Groveler

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN))

        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

E:\>sc qc Groveler

[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: Groveler
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 4   DISABLED
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\WINDOWS\system32\grovel.exe
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Single Instance Storage Groveler
        DEPENDENCIES       : SIS
        SERVICE_START_NAME : LocalSystem

E:\>

However, keep in mind that files already moved into the “SIS Common Store” won’t be moved out and the links (or “reparse points”) will continue to be there even after you have disabled the Groveler.

If you delete the “SIS Common Store” system folder or disable the SIS service (even after you disabled the Groveler service using SISADMIN), you will risk losing data.

There is no SISADMIN option to remove the SIS reparse points and the simplest way to accomplish that is to disable the Groveler and then perform a backup followed by a restore of the entire volume. You will need additional space in the volume as you perform that restore operation, since the previously de-duplicated files will be duplicated again.

Conclusion

SIS is a important feature of Windows Storage Server and Windows Unified Data Storage Server.

With the amount of file duplication created by users these days, you can expect to save a significant amount of disk space.

In fact, a Microsoft IT White Paper on their implementation of SIS pointed to savings of 25 to 40 percent on file servers. That’s hard to ignore…

Links and references

Here are a few additional links about the Single Instance Storage feature of WSS 2003 R2 and WUDSS 2003: