Directory Services Debug Logging Primer

Craig here. In Directory Services we support a whole bunch of components which each have their own debug logging. A while back I pulled together all the information from various KB and TechNet articles and distilled it into a concise list of how logging is enabled for each component we support. I cleaned it up a bit and am posting it here thinking that some of you may also find it useful.

A few caveats before you dive in and start enabling logging all over the place:

  • Enabling debug logging in general is not the first troubleshooting step to take. In many situations it can be overkill and only serve to confuse things. Better to first know the exact steps to reproduce a problem, the exact error messages involved, and overall have a good understanding of the problem end-to-end. I say “in general” because if you are an expert at reading a certain type of debug log, you may be successful at using it earlier in your troubleshooting process, but even that level of expertise rarely gets you around the need for a solid understanding of how the problem presents itself.
  • Not all errors in debug logs are really errors. Many types of debug logging in Windows were never intended to be used by customers to troubleshoot problems, but rather as a way for the developer to debug their own component. Because of that, there are many times when an error is logged that is normal, expected behavior for a component. This is why looking at the debug logs can often only confuse matters if you start troubleshooting an error that is actually perfectly normal behavior for the component. In situations like this it is helpful to compare the output to that of a functioning machine, although even that isn’t always fool-proof at determining what the “real” errors are.
  • Debug logging can be resource intensive and should only be enabled while you are troubleshooting the problem. One of the most common questions asked when we tell a customer to enable some type of logging is “what will this do to the machine?” Unfortunately there isn’t always a simple answer. For many types of logging there is negligible overhead, and the machine will perform nearly the same as if the logging was not enabled. However there are some types of logging that are fairly resource intensive and may cause a tangible performance decrease while enabled. There are many variables that make it difficult to know exactly how it will affect a specific machine. The level of logging enabled, the hardware specs of the machine, and what the machine is being used for are just a few variables that come into play. That is why it is a best practice to enable logging for the shortest period of time necessary to troubleshoot an issue, and then disable it.

Enabling Logging From the Command Line, Vbscript, or PowerShell

Here are a few of the basic methods using command scripting, Vbscript, or PowerShell. Since a majority of logging types are enabled in the registry, the examples will focus on that.

Reg.exe is included in Windows XP, Windows Server 2003, Windows Vista, and Windows Server 2008. It is also part of the Windows 2000 Resource Kit, although that version is not available for public download. Reg.exe lets you manipulate the registry on a local or remote machine from the command line. If the registry key, value name, or value data contain a space, you must put them in quotes. If the value data is a hex number, for example, 0x4B, it must contain the 0x prefix in the command syntax. Using the /f switch with Reg Add and Reg Del prevents it from asking to overwrite if the value already exists, and prevents it from asking for a delete confirmation. To run these against a remote computer, add \<computername> to the registry path, for example, "\SRV01HKLMSOFTWAREMicrosoftWindows NTCurrentVersionWinlogon"

Using Reg.exe to Enable UserEnv Logging

reg add "HKLMSOFTWAREMicrosoftWindows NTCurrentVersionWinlogon" /v UserEnvDebugLevel /t REG_DWORD /d 0x10002 /f

Using Reg.exe to Disable UserEnv Logging

reg delete"HKLMSOFTWAREMicrosoftWindows NTCurrentVersionWinlogon" /v UserEnvDebugLevel /t REG_DWORD /d 0x10002 /f

Using Reg.exe to Determine if UserEnv Logging is Enabled

reg query "HKLMSOFTWAREMicrosoftWindows NTCurrentVersionWinlogon" /v UserEnvDebugLevel

Since the Windows Scripting Host is available on just about every Windows computer you might run into today, Vbscript is another simple way to automate this. To specify a remote computer, you would change strComputer = “.” to strComputer = “SRV01” or whatever the target computer name is.

The first three lines stay the same regardless if you are adding, deleting, or querying the registry value.

Const HKEY_LOCAL_MACHINE = &H80000002
strComputer = "."
Set objReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\" & strComputer & "rootdefault:StdRegProv")

To add the value, you would add this line:

objReg.SetDWORDValue HKEY_LOCAL_MACHINE,"SOFTWAREMicrosoftWindows NTCurrentVersionWinlogon","UserEnvDebugLevel",65538

To delete the value, you would instead add:

objReg.DeleteValue HKEY_LOCAL_MACHINE,"SOFTWAREMicrosoftWindows NTCurrentVersionWinlogon","UserEnvDebugLevel"

To query the value, you would instead add:

objReg.GetDWORDValue HKEY_LOCAL_MACHINE,"SOFTWAREMicrosoftWindows NTCurrentVersionWinlogon","UserEnvDebugLevel",strValue
Wscript.Echo Hex(strValue)

For more information, see WMI Tasks: Registry and the Script Center.

PowerShell is not installed on any released Windows version by default. It is available as an optional component to install in Windows Server 2008, but it is not installed by default there either. Despite that, it really is the best environment for admin scripting, and even with the lack of remoting in version 1.0, we get around that here by using the .NET RegistryKey class, although you could also use WMI’s StdRegProv similar to how it was called in the Vbscript sample above.

The first three lines stay the same regardless if you are adding, deleting, or querying the registry value. Replace SRV01 with the name of your target computer.

$RegKey = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine,"SRV01")
$RegKey = $regKey.OpenSubKey("SOFTWAREMicrosoftWindows NTCurrentVersionWinlogon",$True)

To add the value, you would add this line:

$RegKey.SetValue("UserEnvDebugLevel",0x10002)

To delete the value, you would instead add:

$RegKey.DeleteValue("UserEnvDebugLevel")

To query the value, you would instead add:

"{0:X}" -f $RegKey.GetValue("UserEnvDebugLevel")

The "{0:X}" -f just tells it to display as hex. For more information, see Scripting with Windows PowerShell.

Directory Services Debug Settings

Active Directory Federation Service (ADFS)

To enable debug logging for the ADFS Authentication Package on an account federation server:

Output: %SystemRoot%ADFSLogs
Value Path: HKLMSOFTWAREMicrosoftADFSWebServerAgent
Value Name: DebugPrintLevel
Value Type: REG_DWORD
Value Data: 0xFFFFFFFF

To enable debug logging for the ADFS Token Authentication service:

Output: %SystemRoot%ADFSLogs
Value Path: HKLMSYSTEMCurrentControlSetServicesIfssvcParameters
Value Name: DebugPrintLevel
Value Type: REG_DWORD
Value Data: 0xFFFFFFFF

To enable debug logging for the ADFS ISAPI extension:

Output: %SystemRoot%ADFSLogs
Value Path: HKLMSOFTWAREMicrosoftADFSWebServerAgent
Value Name: DebugPrintLevel
Value Type: REG_DWORD
Value Data: 0xFFFFFFFF

To enable debug logging for the ADFS Web Agent Authentication package:

Output: %SystemRoot%ADFSLogs
Value Path: HKLMSYSTEMCurrentControlSetControlLSAWebSSO
Value Name: DebugPrintLevel
Value Type: REG_DWORD
Value Data: 0xFFFFFFFF

Use the following to specify the level of events that you want logged for Windows NT Token-based applications in the Application log on the Web Server:

Output: Application Event Log
Value Path: HKLMSYSTEMCurrentControlSetServicesIfssvcParameters
Value Name: ADFSEvent
Value Type: REG_DWORD
Value Data: 0x1 (Warning) 0x2 (Information) 0x4 (Success) 0x8 (Failure)

For more information, see Configuring ADFS Servers for Troubleshooting.

AppMgmt

Output: %SystemRoot%DebugUserModeAppmgmt.log
Value Path: HKLMSOFTWAREMicrosoftWindows NTCurrentVersionDiagnostics
Value Name: AppMgmtDebugLevel
Value Type: REG_DWORD
Value Data: 0x4B

Note that you may need to create the Diagnostics key. For more information, see Enable Logging for Software Installation Client Side Extension and KB article 246509.

Certificate Services

Output: %SystemRoot%Certsrv.log
Value Path: HKLMSYSTEMCurrentControlSetServicesCertsvcConfiguration<CA_Name>
Value Name: Debug
Value Type: REG_DWORD
Value Data: 0xFFFFFFFF

CAPI2

CAPI2 is the new cryptography API available in Vista/2008. CAPI2 diagnostics greatly improves on the PKI diagnostics available in 2000/XP/2003. CAPI2 diagnostic information is logged to the CAPI2 Operational log, which is located at Applications and Services LogsMicrosoftWindowsCAPI2Operational in Event Viewer. You can use CAPI2 logging to troubleshoot most PKI operations in Vista/2008.

CAPI2 logging is not enabled by default. To enable it, right-click the CAPI2 Operational log in Event Viewer and select Enable Logging. You can also enable it using Wevtutil:

wevtutil.exe sl Microsoft-Windows-CAPI2/Operational /e:true

To disable it with Wevtutil the syntax is:

wevtutil.exe sl Microsoft-Windows-CAPI2/Operational /e:false

For more information, see Troubleshooting PKI Problems in Windows Vista.

DCPROMOUI

Output: %SystemRoot%DebugDcpromoui*.log
Value Path: HKLMSOFTWAREMicrosoftWindowsCurrentVersionAdminDebug
Value Name: DCPromoUI
Value Type: REG_DWORD
Value Data: 0xFF0003

For more information, see KB article 221254 and Active Directory and Removal Issues.

DFS Replication

Except for event log verbosity, every setting listed below can be enabled using WMIC (included in Windows) or by directly editing System Volume InformationDFSRConfigDfsrMachineConfig.XML. Editing the XML file is more tedious as many settings do not exist in the XML file until they are changed from defaults, so it is not as simple as changing a value, you must also add the associated XML tags. Using WMIC to change the WMI properties takes care of all of that for you.

By default DFS Replication debug logs are created in %SystemRoot%Debug. The log files are named Dfsr#####.log. This naming convention is hard-coded. The log level is set to Informational (4) by default. DFS Replication logs up to a configurable maximum number of 200,000 lines per log file. When the numbers of lines in a log file exceeds the configured maximum lines per log file, DFS Replication will move on to the next log file (Dfsr00002.log, Dfsr00003.log, and so on). In order to conserve disk space (especially since logging is at log level 4 by default), DFS Replication compresses each log file using Gzip compression before creating the next log. Compressed files are named Dfsr#####.log.gz and can be decompressed with most of the common compression tools (Winzip, WinRAR, etc.).

With the default settings (MaxDebugLogFiles=100,MaxDebugLogMessages=200000) all the debug logs combined should use no more than approximately 50 mb of disk space.

Setting: Event Log Verbosity
Output: DFS Replication Event Log
Value Path: HKLMSYSTEMCurrentControlSetServicesDfsrParameters
Value Name: Enable Verbose Event Logging
Value Type: REG_DWORD
Value Data: 0x1

By default the following events are suppressed unless verbose event logging is enabled. This was done to keep the event log free of excessive informational events.

2002 EVENT_DFSR_VOLUME_INITIALIZED
3002 EVENT_DFSR_RG_INITIALIZED
3004 EVENT_DFSR_RG_STOPPED
4002 EVENT_DFSR_CS_INITIALIZED
5006 EVENT_DFSR_CONNECTION_OUTCONNECTION_ESTABLISHED
5004 EVENT_DFSR_CONNECTION_INCONNECTION_ESTABLISHED
   
Setting: Debug Log Severity
Default: 4
Range: 1-5

Sample WMIC command syntax:

wmic /namespace:\rootmicrosoftdfs path dfsrmachineconfig set debuglogseverity=5

Setting: Debug Log Messages
Default: 200000
Range: 1000 to 4294967295 (0xFFFFFFFF)

Sample WMIC command syntax:

wmic /namespace:\rootmicrosoftdfs path dfsrmachineconfig set maxdebuglogmessages=500000

Setting: Debug Log Files
Default: 100
Range: 1 to 10000

Sample WMIC command syntax:

wmic /namespace:\rootmicrosoftdfs path dfsrmachineconfig set maxdebuglogfiles=200

Setting: Debug Log File Path
Default: %SystemRoot%Debug

Sample WMIC command syntax:

wmic /namespace:\rootmicrosoftdfs path dfsrmachineconfig set debuglogfilepath="d:dfsrlogs"

The new path must be created manually; if not, at service restart, the default value %SystemRoot%Debug will be used.

Setting: Enable Debug Logging (NOTE: Debug logging is enabled by default)
Default: True
Range: True or False

Sample WMIC command syntax:

wmic /namespace:\rootmicrosoftdfs path dfsrmachineconfig set enabledebuglog=true

For more information, see Using DFSR.

DFS Management

Output: %SystemRoot%DebugDfsMgmtDfsMgmt.current.log

Steps:

  1. Open %SystemRoot%System32Dfsmgmt.dll.config in Notepad.

  2. Change this line:

    <add key="DfsTraceListenerEnabled" value="0" />

    to this:

    <add key="DfsTraceListenerEnabled" value="1" />

    Change this line:

    <add name="DfsFrsTracing" value="0" />

    to this:

    <add name="DfsFrsTracing" value="65535" />

    Change this line:

    <add name="DfsFrsSnapIn" value="0" />

    To this:

    <add name="DfsFrsSnapIn" value="65535" />

  3. Close and reopen DFS Management (Dfsmgmt.msc) if it was open.

The default maximum log size is 10 MB (10,240 KB). The maximum you can configure it for is 256 MB (262,144 KB). When the maximum size is reached DfsMgmt.current.log is renamed to DfsMgmt.previous.log and a new DfsMgmt.current.log is created.

To change the maximum log size:

Change this line in the %SystemRoot%System32Dfsmgmt.dll.config file:

<add key="MaxTraceLogSize" value="10240" />

to this:

<add key="MaxTraceLogSize" value="262144" />

DNS Client

Setting: DNS Client Service logging
Output: %SystemRoot%System32Dnsrslvr.log

Steps:

  1. Create Dnsrslvr.log in %SystemRoot%System32 with at least one character or space in the file.
  2. Restart the DNS client service.
Setting: DNS Client Registration Services logging
Output: %SystemRoot%System32Asyncreg.log

Steps:

  1. Create Asyncreg.log in %SystemRoot%System32 with at least one character or space in the file.
  2. Restart the computer.

Folder Redirection

Output: %SystemRoot%DebugUserModeFdeploy.log
Value Path: HKLMSOFTWAREMicrosoftWindows NTCurrentVersionDiagnostics
Value Name: FdeployDebugLevel
Value Type: REG_DWORD
Value Data: 0xF

For more information, see KB 907355 and Enable Logging for Folder Redirection Client Side Extension.

File Replication Service (FRS)

Output: %SystemRoot%DebugNtfrs_####.log
Value Path: HKLMSYSTEMCurrentControlSetServicesNtFrsParameters
Value Name: Debug Log Severity
Value Type: REG_DWORD
Value Data: 5
   
Output: %SystemRoot%DebugNtfrs_####.log
Value Path: HKLMSYSTEMCurrentControlSetServicesNtFrsParameters
Value Name: Debug Log Files
Value Type: REG_DWORD
Value Data: 5
   
Output: %SystemRoot%DebugNtfrs_####.log
Value Path: HKLMSYSTEMCurrentControlSetServicesNtFrsParameters
Value Name: Debug Maximum Log Messages
Value Type: REG_DWORD
Value Data: 10000
   
Output: %SystemRoot%DebugNtfrs_####.log
Value Path: HKLMSYSTEMCurrentControlSetServicesNtFrsParameters
Value Name: Debug Log File
Value Type: REG_EXPAND_SZ
Value Data: %SystemRoot%Debug

For more information, see KB article 221112 and