GPMC slow to start? GPO reports failing? You may be missing an index.

  • Article

See if you recognize this:

  • You have lots of OUs in the domain. At least a couple of thousand.
  • Group Policy Management (GPMC) is slow to start. It may take 10 seconds or more, and in extreme cases fails to load at all.
  • Editing a GPO works just fine.
  • Generating a GPO report or executing a GPO backup fails some or all of the time.

One of my customers is a hosting company. They have a complex OU structure for each of their customers, and in total their domain is approaching 100.000 OU's. For a long time they had the symptoms listed above, but of late the problem became unmanageable.

We had a look on a DC using the Active Directory Diagnostics in Perfmon while starting GPMC and generating GPO reports. This undervalued part of Perfmon measures AD activity for 5 minutes and reports all findings, including interesting LDAP queries. One of the more interesting ones was this:

 (&
  (|
    (objectCategory=domainDNS)
    (objectCategory=OrganizationalUnit)
  )
  (gPLink=*)
)

This asks for a Domain object or an Organizational Unit containing a non-zero value of the attribute gpLink. The gpLink is used to reference a Group Policy Object, so this query looks for all objects that have a GPO linked to it. The interesting part was not the query itself, which looks fine, but that the query took more than 10 seconds to execute... an eternity.

So this is a slow LDAP query case. We have a very nice article describing our best practices, called Creating More Efficient Microsoft Active Directory-Enabled Applications. Read it if you are at all interested in LDAP queries. The very first optimization step that the article advises you is to use attributes that are indexed.

This is one of the points where you can see that Active Directory is built on a database. While the AD database is hierarchical and not relational, some of the same principles apply: you can execute queries, and it makes a big difference in performance if you can assume that some of the data you are searching for is sorted. An index takes care of this. So yes, AD has indexes, and attributes that are often used in queries usually have an index of some sort. And yes, you guessed it, the gpLink attribute turns out not to be indexed by default. That could make a huge difference in our query, because out of the thousands of OUs in the domain, you only need the few that actually have GPOs. Another big hint that there was an indexing problem was that the Active Directory Diagnostics report indeed showed that gpLink was not involved in the index used by the query.

Enabling a straight index is simple. The article referenced above actually tells  you how to do it: you look up the attribute needing an index in the schema partition, you take its searchFlags attribute, and enable the first bit (bit zero). Active Directory will pick this up and create an index for the attribute. Each DC does this for itself, meaning that the index itself is not replicated.

In case you missed it: this is a schema change! A reversible schema change for sure, but still, a change. Treat this with the respect it deserves. In 99% of the forests out there this small change does not matter at all. But for a huge enterprise AD with large DIT files and hundreds of DCs all over the world, this is a serious matter. You know who you are.

Armed with this information, we decided to enable the index to see what difference it would make. This can be done using the Schema Management  MMC snap-in, but in this day and age my preferred way is PowerShell. Because this is a one-shot change I kept it very simple. No error checking whatsoever, and you must make sure to meet the prerequisites yourself. The most relevant one is that you must be a member of Schema Admins. The script has no parameters that need to be set.

[powershell]
$dc = (Get-ADForest).schemamaster
$schemaDN = (Get-ADRootDSE).schemaNamingContext
$gplinkDN = "CN=GP-Link,${schemaDN}"

$gplink = Get-ADObject -Server $dc -Filter * -SearchScope Base -SearchBase $gplinkDN -Properties searchflags
if ($gplink.searchflags -band 0x1)
{
Write-Host "attribute gpLink already indexed, no action taken" -ForegroundColor Cyan
} else {
$gplink.searchflags = $gplink.searchflags -bor 0x1
Set-ADObject -Instance $gplink -Server $dc
Write-Host "indexed gpLink, check DCs in the DS log for event ID 1137, attribute gpLink." -ForegroundColor Cyan
}
[/powershell]

The script finds the current Schema Master FSMO, and determines the DN of the Schema Partition. Using that DN, it builds the DN for the gpLink attribute definition. Next, it gets this object with its searchFlags attribute. With a binary and operation it can see if the index is present or not. If not, it sets it and displays a message. Simple enough.

So, if your change manager has approved it and all that, add your admin account to Schema Admins and execute the script. Watch your DCs building the index by checking on event ID 1137 in the Directory Services log. Once the index is done, you can try to see if GPMC behaves any better. In the case of  my customer, the improvement was enormous: GPMC started much faster, and there was no problem anymore generating GPO reports. A  measurement shows that the query time went down to about 100 ms. Not superfast, but much, much better than the original query!

Small word of warning: this was a textbook case. Not all indexing problems are this easily solved, and often the query needs to be changed as well to help the LDAP optimizer pick the correct indices.