Scripts to count and generate statistics about the number of security groups to which an Exchange user belongs


This blog post is a follow-up post to previous 3 CXP flashes that talked about Windows 2003 kernel memory issues and Exchange 2003:


 


The first flash provided technical background about the demands Exchange makes on kernel resources.


 


The second flash discussed hardware configurations that can restrict the kernel memory available for applications.


 


The third flash explained the effect of large user security tokens on Exchange’s kernel memory usage and mentioned the script that can be used to count and generate statistics about the number of security groups to which an Exchange user belongs.


 


This post will introduce the script mentioned above. Scripts themselves can be found at the very end of this blog post.


 


IMPORTANT NOTE: Microsoft provides programming examples for illustration only, without warranty either expressed or implied. This includes, but is not limited to, the implied warranties of merchantability or fitness for a particular purpose. This article assumes that you are familiar with the programming language that is being demonstrated and with the tools that are used to create and to debug procedures. Microsoft support engineers can help explain the functionality of a particular procedure, but they will not modify these examples to provide added functionality or construct procedures to meet your specific requirements.


 


The scripts shown here demonstrate a way to count and generate statistics about the number of security groups to which an Exchange user belongs. This information can be useful in estimating the size in memory of the access tokens associated with users. 


 


Each access token requires a certain amount of Windows kernel memory. The amount varies depending on several factors. Group membership is one of the most important factors, and the size of the token increases in direct proportion to the number of group memberships.


 


If the total token size is under 4K, the amount of kernel memory required for the token is exactly the same as the size of the token. If the token is 2176 bytes in size, it takes 2176 bytes of memory. But if a token goes even slightly above 4K (4096 bytes), then it takes 8K (8192 bytes) of memory. If the token grows to slightly more than 8K, its memory allocation will jump to 12K, and so on. Typically, users who belong to fewer than 80 security groups will have tokens under 4K.


 


Consider a scenario where all your users belong to 79 security groups, and then you add 5 more group memberships for everyone, thus pushing all of them above the 4K boundary. Suddenly, the kernel memory required for each connection doubles from 4K to 8K.


 


An Outlook 2003 cached mode user will typically have 6 to 8 copies of the security token in memory for various connections and operations. A cached mode user with a 4K token therefore needs 32K of paged pool memory for tokens. On a large mailbox server, you will probably have about 200 MB of memory available for connection tokens. Given this, you can support over 6,000 cached mode Outlook 2003 users if tokens are 4K. But with 8K tokens, you can support only 3,000. This is a bottleneck that is independent of other server resources. Disk, CPU and network may all be capable of handling much more load, but none of that matters if you run out of paged pool because of large tokens.


 


Not all security groups that a user belongs to will necessarily be added to the token. The scripts give you a “worst case” result. The rules for which groups get added to a token are somewhat complicated, but I’ll try to give you a summary of the most important points. I’ll refer to the “user domain” and the “server domain.” The user domain is the one in which the user account resides. The server domain is the one in which the Exchange server resides. These may be the same domain.


 


       The rules change dramatically depending on whether you are in mixed-mode or native mode. Switching a domain to native or Windows 2003 functional modes can increase average token size. You should be aware of this possibility before flipping the native mode switch.


 


       Universal, global and domain local groups typically add 44 bytes each to a token, on average. This estimate assumes a SID length of 28 bytes per group and 16 bytes of attribute data associated with each SID. The great majority of SIDs in current and past versions of Windows are 28 bytes in length. You should also count on 500 bytes per token for “overhead” in addition to the space for group memberships. Administrative and specially privileged users are likely to also have larger tokens than average.


 


       Whether the user domain is in mixed or native mode, any global groups to which the user belongs will get added to the token.


 


       If the user account domain is in mixed mode, then you can’t nest global groups in other global groups. But in native mode you can nest, and each nested group and it’s sub-nested sub-groups will be added to the token. Nesting does not help you reduce token size, but it can make it harder for you to figure out how many groups each user really belongs to. (The scripts take nested groups into account, and count them.)


 


       SIDHistory is only available on users in native domains. Groups listed in SIDHistory are added to the token (with duplicate SID suppression). Finishing a migration and getting rid of SIDHistory can make a big difference in token size (your mileage may vary).


 


       If you switch the user domain to native mode, then any universal groups to which users belong (including global groups nested in universal groups) will suddenly be added to the user token, regardless of the mode of the server domain.


 


       In native/Windows 2003 functional modes in the server domain, domain local groups in the server domain will be added to the user token (if the user belongs to any such groups), and nested groups will also be added. If the user account is in Domain A and the Exchange Server is in native Domain B, domain local groups from B will be added to the token. This is why putting servers in a different domain than users can help reduce token size. Most users will belong to more domain local groups in their own domain than in the server domain.


 


Each script has command line parameters and instructions at the top. You can copy and paste the scripts into Notepad and save them as .VBS (not .TXT) files.


 


Groups.vbs simply prints out user names and groups they belong to, separately listing groups from SIDHistory. You can restrict it to a single Exchange server or get a report for multiple Exchange servers using wildcarding. Note: you can’t just use * to get all Exchange servers–you must provide at least a partial server name.


 


Groups_statistics.vbs gives you a text-based histogram view, showing you how many users belong to 50 groups, 60 groups, 70 groups, and so on. If few users belong to more than 80 groups, you likely have tokens under 4K.


 


More information will be available soon in a not-yet-published KB article (article 912376). The article is referenced in the previous blog entry. The article will include information about profiling token usage for various clients and server configurations.


 


groups.vbs script follows:


 


‘==============================================================================


‘ NAME: Groups.vbs


‘ AUTHOR: Kyryl Perederiy, Microsoft IT, MACS Engineering


‘ DATE  : 12/15/2005


‘ COMMENT: The script runs through all mailbox enabled user objects in the


‘ forest and calculates the number of security groups and groups in SID


‘ history for each object. User objects can be filtered by Exchange home server.


‘ PARAMETERS: <output file> <GC Domain Controller> <Domain Naming Context> [<Exchange Server(s)>]


‘ EXAMPLE: CSCRIPT groups.vbs groups.tsv EXCH-DC-01 dc=root,dc=company,dc=com EXCH-MBX-*


‘ Version 1.0


‘==========================================================================


On Error Resume Next


Set strArgs = WScript.Arguments


Set fso = CreateObject(“Scripting.FileSystemObject”)


Set fileStream = fso.OpenTextFile(strArgs(0), 2, True, TristateTrue)


fileStream.WriteLine “DN   Mail   Domain Login  Server GRP    SIDHISTORY”


 


Count=0


DCS = strArgs(1) ‘ Domain Controller


strDomainNC = strArgs(2) ‘ Domain Naming Context for the forest


strFilter = “(&(mail=*)(objectCategory=person)(objectClass=user)” &_


                     “(msExchHomeServerName=*” & strArgs(3) & “))” ‘Mail users search filter


 


Set oConnection = CreateObject(“ADODB.Connection”) ‘ Setup the ADO connection


Set Com = CreateObject(“ADODB.Command”)


oConnection.Provider = “ADsDSOObject”


oConnection.Open “ADs Provider”


Set Com.ActiveConnection = oConnection ‘ Create a command object on this connection


Com.CommandText = “<LDAP://” & DCS & “:3268/” & strDomainNC & “>;” &_


                                  strFilter & “;distinguishedName,mail,sAMAccountName,” &_


                                  “msExchHomeServerName,sIDHistory,homeMDB;subtree”


 


‘ Set search preferences


Com.Properties(“Page Size”) = 1000


Com.Properties(“Asynchronous”) = True


Com.Properties(“Timeout”) = 120 ‘ seconds


set oRecordSet = Com.Execute


 


oRecordSet.MoveFirst


 


While Not oRecordset.Eof


 


       Count=Count+1


       DN = oRecordset.Fields(“distinguishedName”).Value


       Mail = oRecordset.Fields(“mail”).Value


       Server = oRecordset.Fields(“msExchHomeServerName”).Value


       Server = Mid(Server,InStrRev(Server,”=”)+1)


       Domain = Split(DN,”,DC=”)


       Login = UCase(Domain(1)) & “\” & oRecordset.Fields(“sAMAccountName”).Value


      


       set oDirObject = GetObject(“LDAP://” & DCS & “/” & replace(DN,”/”,”\/”))


 


       ‘ tokenGroups is a computed attribute that contains the list of SIDs


       ‘ due to a transitive group membership expansion operation on a given user


       oDirObject.GetInfoEx ARRAY(“tokengroups”),0


      


       ‘ Size of the array correspond to the number of groups


       GROUPS = ubound(oDirObject.GetEx(“tokengroups”))+1


 


       If IsNull(oRecordSet.Fields(“sIDHistory”).Value ) Then


              SIDHIST = “0”


       Else


              SIDHIST = ubound(oDirObject.GetEx(“sidhistory”))


       End If


 


       WScript.Echo Count & CHR(9) & DN & CHR(9) & GROUPS


       fileStream.WriteLine _


              DN & CHR(9) &_


              Mail & CHR(9) &_


              UCase(Domain(1)) & CHR(9) &_


              Login & CHR(9) &_


              Server & CHR(9) &_


              GROUPS & CHR(9) &_


              SIDHIST & CHR(9)


 


       oRecordset.MoveNext


 


Wend


 


WScript.Echo “Total: ” & Count & ” users found on the server(s): ” & strArgs(3)


 


groups_statistics.vbs script follows:


 


‘==========================================================================


‘ NAME: groups_statistics.vbs


‘ AUTHOR: Kyryl Perederiy, Microsoft IT, MACS Engineering


‘ DATE  : 12/15/2005


‘ COMMENT: Runs through all mailbox enabled user objects in the forest and


‘ calculates statistical distribution for group membership


‘ PARAMETERS: <output file> <GC Domain Controller> <Domain Naming Context> [<ExchHomeServerName>]


‘ EXAMPLE: CSCRIPT groups_statistics.vbs groups_statistics.tsv EXCH-DC-01 dc=root,dc=company,dc=com EXCH-MBX-0*


‘ Version 1.0


‘==========================================================================


On Error Resume Next


Dim GROUPS(100)


Set strArgs = WScript.Arguments


Set fso = CreateObject(“Scripting.FileSystemObject”)


Set fileStream = fso.OpenTextFile(strArgs(0), 2, True, TristateTrue)


fileStream.WriteLine “Groups” & CHR(9) & “Users”


 


Count=0


DCS = strArgs(1) ‘ Domain Controller


strDomainNC = strArgs(2) ‘ Domain Naming Context for the forest


strFilter = “(&(mail=*)(objectCategory=person)(objectClass=user)” &_


                     “(msExchHomeServerName=*” & strArgs(3) & “))” ‘Mail users search filter


 


Set oConnection = CreateObject(“ADODB.Connection”) ‘ Setup the ADO connection


Set Com = CreateObject(“ADODB.Command”)


oConnection.Provider = “ADsDSOObject”


oConnection.Open “ADs Provider”


Set Com.ActiveConnection = oConnection ‘ Create a command object on this connection


Com.CommandText = “<LDAP://” & DCS & “:3268/” & strDomainNC & “>;” &_


                                  strFilter & “;distinguishedName,sAMAccountName;subtree”


 


‘ Set search preferences.


Com.Properties(“Page Size”) = 1000


Com.Properties(“Asynchronous”) = True


Com.Properties(“Timeout”) = 120 ‘seconds


set oRecordSet = Com.Execute


 


oRecordSet.MoveFirst


 


While Not oRecordset.Eof


 


       Count=Count+1


       set oDirObject = GetObject(“LDAP://” & strArgs(1) & “/” &_


              replace(oRecordset.Fields(“distinguishedName”).Value,”/”,”\/”))


       oDirObject.GetInfoEx ARRAY(“tokengroups”),0


       GRP = ubound(oDirObject.GetEx(“tokengroups”))+1


       GROUPS(Int(GRP/10)) = GROUPS(Int(GRP/10)) + 1


       WScript.Echo Count & CHR(9) & oRecordset.Fields(“sAMAccountName”).Value & CHR(9) & GRP


       oRecordset.MoveNext


Wend


WScript.Echo “Total: ” & Count & ” users found”


WScript.Echo “See ” & strArgs(0) & ” for details…”


For i=0 to 100


       fileStream.WriteLine i*10 & CHR(9) & GROUPS(i)


Next


 

Mike Lee

Comments (11)
  1. Brian says:

    Ok, appreciate the scripts but I can’t seem to get them to run. I am (sadly) not a scripting person so maybe I’m doing something wrong… but I cut and pasted into notepad, saved as vbs, ran the command as documented (starting with cscript…), and on both scripts I receive the error message:

    groups.vbs(42, 1) Microsoft VBScript compilation error: Syntax error.

    (of course it has the other script name for the other script).

    Can anyone point me in the right direction? The only thing I’m not sure of is whether I’m doing the domain naming context correctly, but I think I am. For example, if our domain is tom.dick.harry.com, i’m using dc=tom, dc=dick, dc=harry, dc=com for that. Thanks in advance for any tips!

  2. Al says:

    In Response to Brian. When you copy from the web, it often puts in formatting that you don’t see displayed. In your case, I believe if you look, you’ll see a bunch of spaces in your notepad version of the script. Take out the spaces and try again. If you still get an error, open notepad, and use CTRL+G to go to the line referenced in the error. If it got line wrapped or a space or other character got entered, it needs to be fixed for formatting. Keep up this cycle and you’ll get it working.

    Your grasp of domain context (dc=tom,dc=dick,dc=harry,dc=com) is correct.

    -ajm

  3. Darren says:

    You need to edit the .vbs files, since every line has a blank space when the files got posted, the continuation vb lines cause problems with multiline variable declaration. You need to remove the blank line in between the declaration.

    In groups.vbs, remove blank lines from the following multiline variable values:

    – strFilter

    – Com.CommandText

    – fileStream.WriteLine (near end of while loop/wend)

    In groups_statistics.vbs, remove the blank lines from the following multiline variable values:

    – strFilter

    – Com.CommandText

    – Set oDirObject

    Once doing this, both mine ran fine.

  4. Teo Heras says:

    Mike,

    Thanks for the great article! It’s something I’ve never come across before. Do chached mode clients require more or less paged pool memory from the server than non-chached mode clients?

    Teo

  5. Brian says:

    Great, works now! Thanks Al & Darren for the assist!!

  6. Exchange says:

    Thank you Darren for helping on this!!

    Apologize for the confusion on this – we do realize there are issues with pasting scripts into the body but that’s the only way we can put them up on the blog so they download to RSS readers. When KB article gets published – scripts will be available in the form that does not need tweaking.

  7. joe says:

    I think you will find you can substantially speed up these scripts by dumping the query

    strFilter = "(&(mail=*)(objectCategory=person)(objectClass=user)" &_

    "(msExchHomeServerName=*" & strArgs(3) & "))" ‘Mail users search filter

    and instead looking at values of the homeMDBBL attribute on the msExchPrivateMDB objects of the server in question.

    The medial portion of the filter above (the msExchHomeServerName=*servername) can be quite slow unless you have enabled medial indexing on that specific attribute. You should seriously notice a difference in the two mechanisms of retrieving the user DNs with mailboxes on that server if you have busy DCs or a large domain.

    The query you show would have to look at the very least at every object with a mail address as the mail attribute would most likely have the smallest number of objects of the indexed attributes in the query. Lots of contacts or mail enabled groups would add even more time to the query plus if you have lots of Exchange servers you have to weed out all but the one you are interested in. So you probably end up making AD touch as many objects as have mail addresses, hundreds, thousands, tens or hundreds of thousands, etc.

    On the flip side, looking at the homemdbbl values on the private MDB objects shows you exactly who has mailboxes on that server and will only have to touch and return maybe 20 objects.

    Another alternative is to create a normal (non-medial) index for msExchHomeServerName and use the full value of the name instead of doing a medial search. That still probably wouldn’t be as fast as looking at the private mdb objects but I would have to test it to be sure.

    joe

  8. Mike Lee says:

    @Teo Heras

    Yes, Exchange cached mode uses more paged pool than online mode. I’m still working on providing more detail and recommendations for a KB article, so take what I say here as preliminary only.

    An online user will typically cause 3 or 4 copies of the token to be generated, while a cached mode user requires 6 to 8 copies. This can go up if you have public folder and free/busy access happening on the mailbox server.

    Here’s a quick and dirty way to estimate how many copies a particular Outlook 2003 user is generating (this will typically estimate a little high):

    Ctrl+right click the Outlook 2003 icon in the system tray. This exposes the Connection Status dialog. On that dialog, count the number of connections listed for a particular server.

    Now, run netstat -o and count the connections going to the Exchange server.

    Add these together and add 2, and that gives you a decent estimate of number of connections.

    The logons page in ESM does not give you an accurate count, for this purpose.

  9. Kyryl Perederiy says:

    Joe,

    Thanks for the advice. You are right, the performance of the script could be improved in several ways at the cost of flexibility. In large environments you’re typicaly will be interested in looking at a bunch of servers (like EXCH-MBX-*). And yes, it will take several seconds or minutes to execute a query but you want to run it only once.

    Kyryl

  10. Ed says:

    Do these scripts include the group membership of all DLG’s in the Forest, or only the DLG’s of the domain in which the script is run against?

  11. Kyryl Perederiy says:

    Ed,

    The second is true. These scripts will count only DLG’s of the domain in which the script is running.

Comments are closed.