An adventure in the sccmcontentlib – single instance store


Sometimes I have nightmares which are work related. One such nightmare was about Configuration Manager 2012 and it prompted me to write this blog post. In my dream, the telephone rang and it was my Configuration Manager 2012 Distribution point. It kindly told me that one of my packages disappeared for no reason. Geez I need a vacation.

With SCCM 2007, content is stored on a distribution point in package ID folders. If you knew the package ID, you could browse to the file share and install/run manually.

This option is more complicated now in Configuration Manager 2012. The files are not just stored on a drive in a manner you could just copy and execute.

The following blog post is about the new Content Library and my attempts to reconstitute a package from the Single Instance Store.

By default, the content is single storage and browsing to a DP does not store all the files in a package id folder where you could find an install and execute it. You can enable the older method which will create a separate folder share. This will require double the disk space! Obviously, this is a pain point for those who use the distribution points this way as a mitigation step in a crisis.

To understand how the Content Library works in Configuration Manager 2012, I decided to find a way to solve this pain point.

This functional diagram is my review of the single content library:

 

Content Layer

In the Content Layer, individual files are sent and only stored once in a single storage location. For example, there might be three packages that need to be distributed in a Configuration Manager 2012 infrastructure. All three packages contain the same file Psexec.exe with the same md5\sha1\sha2 hash value.

Why send the file and store it in three places?

Great efficiencies are gained with the new shared single storage Content Library in Configuration Manager 2012. The file is hashed, given an ID and sent only once. Each package is stored as pointers to the necessary files. This means that packages share files if they are the same. This saves on network usage and file storage requirements. Sounds awesome; doesn’t it!

Configuration Manager Layer

Here is an example:

I created two packages:

  1. 1.      Filezilla – package id HH00005
  2. 2.      Notepad ++ - Package id HH00006

Each has psexec.exe, one of my favorite sysinternal tools. Note you can get it here at http://live.sysinternals.com

Next I “Distribute Content” in the CM2012 console to my distribution point.

Distribution Point Layer

When content is sent to the distribution point, it is stored by default on the largest available drive in a folder, sccmcontentlib. During the install you can set this location, however, you cannot configure the drive settings in the distribution point properties after installation completes.

The following folders get created:

  • ·        Datalib = Data Library
  • ·        Filelib = File Library
  • ·        Pkglib = Package Library

DataLib

For each package that you distribute a corresponding package+id.version number folder and file will be created. For patches the folder will appear as the patch guid and application folders will appear as Content_guid

Now let’s look at the package HH00005.1 (Filezilla)

Each DataLib folder contains .ini file which includes important information for the actual file.

  • ·        Attributes
  • ·        Size
  • ·        TimeModified
  • ·        Hash

 Viewing the psexec.ini file in the HH00005.1 package folder:

 

A Hash value of F8DB… is found. The first four entries of this hash are important. This is the locater folder name for the actual binary file in the pkgLib folder.

Note

The default hashing algorithm for content is SHA-256

To find the file:

Filelib

The folder names have meaning. Each corresponds to a hash value in the FileLib folder. In the example of HH000005.1, psexec.exe.ini, the value was F8DB

Viewing the F8DB folder:

The hash value .ini contains a list of the type and references of which packages use this file. Notice that there are two packages that are listed. This is single instance storage. Any package/program/content/update that has this exact hash file requirement will use this file and location.

The actual file will be the hash with no extension. Copying and renaming the file to psexec.exe, I verified that is the psexec file:

SHA-256 hash value matches.

Now back to the HH00005 package. It contains the setup for FileZilla and psexec.

Note that the psexec.exe.ini file points to the F8DB filelib folder as I demonstrated above.

To summarize the Content Library, Single Content Storage and this review:

Both HH00005 and HH00006 CM2012 packages contain the file psexec.exe. The file is only sent once to the Content Library and stored as single instance. A SHA-256 hash value is used to track the unique file and the first four digits are used as a locator. When a client requests the download the files are assembled from the Content Library via single instance storage.

Now suppose you want to reconstitute an entire package from the sccmcontentlib folder. I created a basic vb.net application to assist me in this exercise.

Following each PkgLib file to the DataLib, I was able to regenerate all the files from the FileLib for any given package.

I hope my blog posting was helpful in providing an understanding the Content Library and the Single Instance Storage in Configuration Manager 2012.


Comments (22)

  1. Anonymous says:

    Excellent walk through of how the single-instance store works. Thanks!

  2. Anonymous says:

    Sure . I’ve since moved it to PowerShell but I will clean both up and post it back to this link.

  3. Anonymous says:

    AGREE with the previous comments, Howard! THis is the best I've yet to find that explains this quite complex process in detail as you've done. Thans much for this!!!

  4. Anonymous says:

    Any chance we can get the VB app you made ?

  5. Anonymous says:

    Between Hodads and those darn breakfast burritos – I'll need more than a vacation. Love the weather in San Diego!! Looking forward to working the summer with you.

  6. Anonymous says:

    hi,
    thanks for this post , for me its become clearer .Any news about vb or powershell app ?

  7. Jeff25 says:

    Howard,

    You are just finishing up that gig in Houston and are back working with me in San Diego. You are having a vacation. We need to make a trip to Hodads to get rid of these CM 2012 nightmares and get you dreaming about food.

  8. MMears says:

    I can't believe you're not going to share your app with us . C'mon, pretty please!

  9. Joachim says:

    I have some packages were no packageid.ini files is listed on the distribution point. What can be the cause or is it by design?

    Thanks

  10. s says:

    Do you know if SCCM 2012 provides an API to "recompile" this?

  11. The Solver says:

    Hi Joachim

    Sorry it it's too late to ask about your package issue…

    The package object has a correct Package Source Path? You're searching the missing  packageid.ini files on a DP, if you are searching on DP… Are you sure that the Package content are distributed on the selectect DP, I try to update the content to see it the content library refresh the changes. And the last trick… pay attention on the monitoring – System Status – all message querys (All messages) may be the problen are logged.

    Samu

  12. Jeff Kilmer says:

    Howard….this is an excelllent post on explaining how the content lib sis functions.  Great Job!

  13. Gregory Wood says:

    I was really hoping this would be *true* binary / block-level single-instance storage.   I don't see how this really would help much at all.  saving a few hundred kilobytes on a copy of psexec isn't much.  You'd have to have an extraordinarily large SCCM file structure to have enough identical files for it to really make a difference, and I'd still say it's going to be very little percentage. 🙁

  14. Anonymous says:

    ~ Bindusar Kushwaha Hi everyone, here in product support we regularly see issues relating to how the

  15. .ht says:

    Great post Howard.. very useful..
    Hey.. is there any chance that you share the vb.net application to reconstitute packages with us?

  16. Brent says:

    What does it mean if DataLib, FileLib and PkgLib have not been created inside the SccmContentLib folder after reinstalling the DP role?

  17. JMO says:

    Anyone finding an alternative link to the tool (VB, powershell, etc.) which is presented above, please post it… I’m having no luck locating something…

  18. laurent says:

    hi brent ,
    in this case yout must push the dp role by sccm console

  19. laurent says:

    hi JMO
    hi I asked my PFE to find it. I will send it as soon as I
    laurent

  20. Darren says:

    Does anyone have a link they can share to the powershell or vb tool?

  21. Daniel says:

    Hi , great post. For those asking for a tool to download package content I would recommend downloading using the web browser. You will need to know the distribution point server name and package ID. Once you have this simply type this in the address bar
    replacing DP server name and package ID: http://dpservername/sms_dp_smspkg$/packageid

  22. Kim Aller says:

    I tried this tool but cannot find any link to download vb script/powershell

    Thanks,
    Kim Aller,
    https://techsupportphonenumber.net/mcafee-contact-phone.html

Skip to main content