Using PowerShell for Message Analyzer Text Log Parsers

Using PowerShell for Message Analyzer Text Log Parsers

Hi Everyone, Brandon Wilson here with you again to discuss how to use PowerShell to generate parsed data output for log files for Message Analyzer (otherwise known as .matp files). Today we will focus on the text log parsers available as of Message Analyzer 1.3.1, so depending on the version you are running, you may or may not see all of the text log parsers mentioned within this blog. I HIGHLY recommend updating to Message Analyzer 1.3.1 as soon as possible though, as there are a lot of improvements to be had!

Before I get started, it’s important to remember that within Message Analyzer there are a few different feedback mechanisms for you to use (and we want to see your feedback). You can use the feedback link/button in Message Analyzer, reach out in the Message Analyzer forum, or you can leave a comment here, and I will pass it along (I promise). For the Netlogon parser, you can also send an email to MANetlogon@microsoft.comwhere you can submit ideas and any problems or buggy behavior you may run across. I highly recommend reaching out through one of the available methods to provide your suggestions and input!

You can also read up more on Message Analyzer itself at http://blogs.technet.com/MessageAnalyzer

In this walkthrough, we will cover the following:

Saving your output through the GUI

Available text log parsers

Known issues

What commands do I need?

Opening your output

Reference links

Saving your output through the GUI

Once you open your log file within Message Analyzer directly, you have the ability to save the output as a .matp file. What this does is generate an output file, with the .matp extension, that is already parsed. This provides two major advantages:

1. You don’t have to wait for parsing to complete if you need to review/analyze the file again (or provide it to someone else)

2. In most cases, the file size is reduced vs. the original input file

 

This is a pretty simple thing to accomplish. Here’s a quick rundown:

1. Open Message Analyzer

2. Open the log file you wish to parse (I will use a Netlogon log in this example)

image

3. Click the Save button in the top ribbon

image

4. In the save window that pops up, select either “All Messages” or “Filtered Messages for Analysis Grid view” (depending on if you want to save just a portion you’ve filtered which is showing in the analysis grid or if you want to save all messages). After making that selection, click the “Save As” button in order to save the file as a .matp file.

image

5. Specify a path and file name for the output file (the default will be the session number; for example, “Session 1.matp”)

 

And that’s all there is to it…

 

 

Available text log parsers

There are a number of text log parsers available for your use. I am not going to cover these in depth within this blog, but I did want to provide you with the options you have available.

image

 

So even though its shown above in the screen shot, let’s take a look at what we have here. Most of these are pretty self-explanatory:

1. CheckSymCSV

2. Cluster

3. Dcdiag

4. DPMRegistry

5. EventLogCSV

6. IIS

7. LSP

8. LYNC

9. Netlogon

10. SambaSysLog

11. SCCM

12. SQLServerError

13. SQLServerSetup

14. ULS

15. VMM

16. DefaultSimpleLogFileReaderConfig

 

 

Known issues

There are a couple of known issues that have occurred over time, and to be honest, I don’t know that I would actually call them issues (except for the first one).

1. There was a regression in Message Analyzer 1.3 that prevented PowerShell text log parsing from functioning properly. This has been fixed in Message Analyzer 1.3.1.

2. Parsing performance may be a little slower in PowerShell vs. using the GUI. This is in part because PowerShell is not only parsing the data, but it is also saving the .matp output file.

 

 

What commands do I need?

For this example, we will use a Netlogon log named DC1-netlogon.log. Fortunately, parsing in PowerShell is pretty straightforward, even for me. In the example, I will show you a way to run the necessary PowerShell commands and pipe it out to a file the old school way, as well as log the performance/runtime of the command itself (just in case you’re interested in such things). Just as a reminder, you can save any of these via the PowerShell ISE, or even via notepad, as a .ps1 file. That file can then be ran either through the PowerShell ISE, or via the PowerShell prompt by using the “.\scriptname.ps1” syntax (no quotes). The same methods below apply to all the text log parsers available through Message Analyzer.

So, what do we ACTUALLY need to do (in the barest form):

1. First, we need to specify a trace session and provide a path for the .matp output

2. Second, we need to provide the path to the log file, as well as the text log parser we want to use

3. Third, we need to actually run the command to begin parsing using the input provided in steps 1 & 2

 

So, a simplistic example could be this:

$TraceSession01 = New-PefTraceSession -Force -Path B:\temp\DC1-Netlogon.matp -SaveOnStop

Add-PefMessageSource -PEFSession $TraceSession01 -Provider "B:\temp\DC1-Netlogon.log" -LogConfig Netlogon

Start-PefTraceSession -PEFSession $TraceSession01

 

That’s it; nice and simple right!

 

Now, let’s say we want to track the performance and write the performance details to an output file (which is handy if you running commands against multiple files, or of course, if you’re just curious).

So, showing you a view of the PowerShell ISE for just one example against a single Netlogon log:

image

 

As you can see in the above example, I am doing a basic echo command to tell me which file (in this case, which domain controller) I’m parsing and providing that to a file named Netlogon-tracking.txt located in the B:\temp directory. Next, because I like to know my start (and end) times, I do a simple get-date PowerShell command to output the date and time when the command started (and ended). At that point I kick off the actual meat of the script, except unlike the basic example, I threw a measure-command PowerShell command into the mix in order to evaluate the run times when the script is actually kicked off. After all of that, I simply throw a couple of blank lines in to get ready for any other data I might want to parse as well.

Obviously, all of these extras are optional and aren’t really needed, but I wanted to show you some of the flexibility here.

 

For reference, this is the type of output that the “measure-command” PowerShell command will get you:

DC1-Netlogon.log…

Friday, July 24, 2015 11:07:41 AM

Days : 0

Hours : 0

Minutes : 0

Seconds : 18

Milliseconds : 660

Ticks : 186609256

TotalDays : 0.000215982935185185

TotalHours : 0.00518359044444444

TotalMinutes : 0.311015426666667

TotalSeconds : 18.6609256

TotalMilliseconds : 18660.9256

Friday, July 24, 2015 11:08:06 AM

 

Now, let’s say I want to expand on this parsing and review 2 domain controllers and an Exchange server’s Netlogon log. Basically, this is the same as above, except we are calling out more files. Let’s take a look:

image

 

 

Opening your output

Opening the .matp output file is as simple as can be. All you need to do is double click the file to open it, or of course you can right click and select Open (or Open With), or you can open it in Message Analyzer directly using the File menu. Whatever method you choose, when you open the output file, the Message Analyzer GUI will open, and you will be presented with the already parsed output file (as seen in the below screenshot).

image

 

 

Reference links

 

Message Analyzer v1.3.1 download (highly recommended!)

http://www.microsoft.com/en-us/download/details.aspx?id=44226

 

Using PowerShell to Automate Tracing

http://blogs.technet.com/b/messageanalyzer/archive/2013/10/29/using-powershell-to-automate-tracing.aspx

 

Diving into the Netlogon Parser (v3.5) for Message Analyzer

http://blogs.technet.com/b/askpfeplat/archive/2015/08/03/diving-into-the-netlogon-parser-v3-5-for-message-analyzer.aspx

 

New Features in the Netlogon Parser (v1.1.4) for Message Analyzer

http://blogs.technet.com/b/askpfeplat/archive/2015/01/19/new-features-in-the-netlogon-parser-v1-1-4-for-message-analyzer.aspx

 

Introducing the Netlogon Parser (v1.0.1) for Message Analyzer 1.1 (By: Brandon Wilson)

http://blogs.technet.com/b/askpfeplat/archive/2014/10/06/introducing-the-netlogon-parser-v1.0.1-for-message-analyzer-1.1.aspx

 

Troubleshooting Basics for the Netlogon Parser (v1.0.1) for Message Analyzer (By: Brandon Wilson)

http://blogs.technet.com/b/askpfeplat/archive/2014/11/10/troubleshooting-basics-for-the-netlogon-parser-v1-0-1-for-message-analyzer.aspx

 

Quick Reference: Troubleshooting Netlogon Error Codes (By: Brandon Wilson)

http://blogs.technet.com/b/askpfeplat/archive/2013/01/28/quick-reference-troubleshooting-netlogon-error-codes.aspx

 

Quick Reference: Troubleshooting, Diagnosing, and Tuning MaxConcurrentApi Issues (By: Brandon Wilson)

http://blogs.technet.com/b/askpfeplat/archive/2014/01/13/quick-reference-troubleshooting-diagnosing-and-tuning-maxconcurrentapi-issues.aspx

 

Message Analyzer Forum

http://social.technet.microsoft.com/Forums/en-US/home?forum=messageanalyzer

 

Message Analyzer blog site

http://blogs.technet.com/MessageAnalyzer

 

Memory usage with Message Analyzer

http://blogs.technet.com/b/messageanalyzer/archive/2015/07/06/memory-usage-with-message-analyzer.aspx

 

 

Just to recap; please send us any suggestions or problems you identify through the comments below, the Message Analyzer forum, or using the integrated feedback button in Message Analyzer as seen below (circled in green at the top right)!

image

 

image

 

 

That about sums up today’s blog. Can you believe I wrote something this short (I know I hardly can)! Thanks for reading, and as always, I’ll talk to you next time around!

 

-Brandon Wilson