DFSR Debug Analysis with Message Analyzer – Part 7, Dealing with Message Analyzer Limitations


This post continues the series that started here.

Default Parser

Up to this point the development of my parser was tested against a single DFSR debug log file. Once I started loading multiple DFSR debug log files, I ran into an issue. I had to choose my parser for every log file I want to parse in the same session –

MA11

I discussed this with Paul Long and he was able to provide a terrific solution. I create a file called TextLogConfigMapping.txt with the following contents –

Dfsr*.log = DFSRDebug.config

This simply states that any file name matching Dfsr*.log will use the parser DFSRDebug.config.

I place this file in %LOCALAPPDATA%\Microsoft\MessageAnalyzer and now when I load multiple files, the New Session view defaults to –

MA12

Continuous Log Files

After making the change discussed above, I thought I was in a great place to process large numbers of DFSR debug log files together as if they were a single file. Unfortunately there are a few issues –

  1. Multi-line messages may span two log files (message begins in log file X and continues in log file X+1)
  2. Message Analyzer does not treat consecutive files as a continuous log
  3. Every DFSR debug log file begins with a log header and Message Analyzer does not have a method for ignoring messages (every message must be parsed)

The result is that parsing breaks for multi-line messages that span files. To address this, I’ve decided to pre-process my DFSR debug logs at the command line as follows –

copy dfsr*.log combined.log

findstr /v /b /c:"*" combined.log > dfsr.log

This

  1. Copies the contents of my DFSR debug log files into a single file called combined.log
  2. Extracts all lines not starting with * and puts them in dfsr.log

I’m left with a single file that my parser cleanly handles.

The other issue that arises is that if I’m interested in analysis of just one file that starts in the middle of a multi-line message, the first few lines won’t parse correctly. I’ve chosen to handle this by parsing lines that start with a + with –

/////////////////////////////////////////////////////
// Broken Multi-line continuation
//     These may occur at the beginning of a debug
//     log if the log rolled over while writing a
//     multi-line message
/////////////////////////////////////////////////////

message DfsrMultiLineMessageContinuation with
    EntryInfo { Regex = @"(?<MessageText>\+.*)", Priority = 1 },
    DisplayInfo { ToText = GetHeaderSummaryText } : LogEntry
{
    string Annotation = "Multi-line continuation from a previous debug log";
    string MessageText;

    static string GetHeaderSummaryText(any d)
    {
        var e = d as DfsrMultiLineMessageContinuation;
        return e.MessageText;
    }
}

Similar to DFSR debug log headers, these lines are injected entirely into a MessageText field and annotated for easy filtering.

Complete Parser

After making the updates in this post and adding a few other specific multi-line message types, v1 of my parser is complete. You may download it here –

DFSRDebug.config

Parser Performance

After combining 323 DFSR debug log files using the method described above, I have a single 3.93 GB log file which Message Analyzer parses in about 20 minutes.

Once parsing is complete, I save the session as a .matp file, reducing file size to 0.99 GB. This file then loads in about 4 minutes with all parsing included – around 8.5 million messages.

Next Up

DFSR Debug Log Charts

Comments (2)

  1. JamieB says:

    I wanted to pass on a PowerShell equivalent for your command shell log pre-process process. PowerShell appears to have no way to combine files like the command shell did, so it’s a little less intuitive what needs to happen. However, since the log headers
    are always just the first two lines of the log file (to my knowledge), we can strip them out inline. Here are two versions (assuming you are in the folder with the logs):

    # Create ’empty’ output file
    Out-File -FilePath DfsrCombined.log
    # copy the content, skipping the first two lines of each file
    Get-Content -Path DFSR0*.log | Select-Object -Skip 2 | Add-Content -Path DfsrCombined.log

    An alternative method to ensure no lines with a leading ‘*’ are included (removes the 2 line assumption):

    # Create ’empty’ output file
    Out-File -FilePath DfsrCombined.log
    # copy the content, skipping the first two lines of each file
    Get-Content -Path DFSR0*.log | Where-Object { $_.Trim() -NotMatch "^*" } | Add-Content -Path DfsrCombined.log

    In both cases, interactively you can shorten the command lines dramatically with aliases.

    As a side note, this can also be done with Out-File, but in some limited testing, I found the Add-Content method was about 50% faster than Out-File.

    Jamie

Skip to main content