Announcing the Message Analyzer 1.1 Release!


We’ve been working hard and are proud to announce Message Analyzer 1.1. The list of new features are summarized below, but it’s impossible to list everything. So stay tuned for more details from the blog as we add more information to help you discover how to most effectively use the new version.

You can find more details about Message Analyzer 1.1 on our page on the download center, or click the links below to download now.  Warning: upgrade will overwrite the AppData/local folder and OPN or Config changes will be lost.  Please backup your work before upgrading!

I’ve also record a quick 12 minute video to provide an Introduction to Message Analyzer, which should help you get started if you are new, and provide some tips if you are already familiar.

Summary of Features:

  • Improved Performance – You should see a significant performance increase. Both in work flow and raw processing. In some cases we still need to build caches for parsing, but after that one time hit, the second time will be faster than ever.
  • More Built-in Assets – Included now are new charts, filters, trace scenarios, sequence expressions and more. For instance you’ll find a Top Talkers, TCP Steven’s Chart and an SMB performance summary chart.

image

image

  • Multiple Remote Capture – Capture from multiple machines on your network at once, and bring all the trace data, including other ETW providers, locally for analysis. Includes support to configure and capture from PowerShell remotely.

image

  • New Session Workflow – We heard the feedback about starting new trace sessions and importing data so we’ve improved that workflow significantly.
  • MOF/WPP ETL – We support the parsing of MOF and WPP based ETL files.
  • TLS/SSL Decryption – Add your private certs via the option page and decrypt your TLS/SSL data seamlessly. Get feedback, using the new Decryption tool window which tells you which TLS/SSL conversations were decrypted. Select the conversation in the Decryption tool and the related messages are selected in the Analysis grid and other views. Notice in the visual stack tool how HTTP is decrypted above TLS.

image

  • Aliases – No longer memorize arcane IP addresses and instead replace them with friendly names that you can save, share and quickly turn groups of aliases on and off.
  • Time Improvement – Display the Time only, and change your relative time zone so you can understand your data from the customer’s perspective.
  • Sequence Expression Updates – Now you can also defined formatted output with sequence expressions so each match provides a useful summary for each result. For instance, run the TCP 3-way sequence

image

  • Updated Parsers – Again we have continued to increase the number of prioritized parsers available, not to mention the new Office parsers.
  • Updated Call Stack – This updated tool window now has 4 different operation modes including this visual stack.

image

  • ResponseTime – A new property for an operations, which combines requests and responses, and computes the response time, or how long it takes between the last part of the request to the first part of the response. This measures the responsive of the service, compared to the Time Elapsed which includes the delay due to the network.

image

  • Experimental Features – We have some hidden features with partially baked ideas that we love to hear feedback about. Feel free to enable, play and tell us about these new tools we are working on.

image

We are excited about all the new features and improvements and hope you will be too. And with that, there’s nothing left but to download it now and give it a test drive. And as always, we value your feedback, which by the way, is now possible by clicking the smiley face in the upper right hand corner.

Enjoy!


Comments (63)

  1. Paul E Long says:

    Piotr, good point on the losing changes. While this will affect a limited number of users, those users will be surprised. I’ve added a warning.

    As for your time range scenario, we’ve supported this for some time. In fact we put some focus on this idea and continuing to expand this scenario. Additionally, this is where we support combining multiple sources so you can correlate between multiple logs.

    To load a subset of data, you go to the File->New Session menu, then chose the Files option. Next Add Files, you can add traces, text logs, ETLs, Event Logs and other things here. Now you get time sliders that let you narrow down the range. This option is greyed
    out until you select the check box to limit the start or end. If you don’t have the option to use the sliders, probably because we can’t detect the start/stop of every type of file, you can manually type in a time using the YYYY-MM-DDTHH:MM:SS.FFFF format
    (ISO 8601). You also have the option to add a filter to further limit the amount of data you pull into memory.

    The time parsing can be fast. If we are talking about structured logs, we don’t have to parse, so we can more quickly limit the data we pull in. However filtering, and non structure logs require that we parse each frame. You won’t gain any time, but your memory
    will be spare.

    Paul

  2. Graeme Bray says:

    Any chance of this being an available product to update via WSUS like NetMon was? It would be nice to use this to update the existing installs of Message Analyzer that we have on systems without having to do it manually.

  3. Paul E Long says:

    @Graham, the only documentation right now is the parsers which basically describe the protocol programmatically. The file is WFPCapture.opn and is located in your appdatalocalMicrosoftMessageAnalyzerOPNandConfigurationOPNs directory. You can also
    right click a field in the Details window and Go To Definition.

    With loopback captures, you need to configure them to capture in one direction. The Loopback scenario, included with Message Analyzer, should do this automatically and then filter only loopback addresses. This should insure you see the traffic only once.

  4. Paul E Long says:

    @Jon, yes there were very good reasons. Among them, more flexible parsing to handle HTTP and XML, Validation of protocol behavior, better transparent support for reassembly, inbox capture drivers and so much more. But we are certainly trying to bring back
    the missing features, so I you have a favorite let us know.

  5. Paul E Long says:

    Yes, we did address that bug. Feel free to let us know if we can still improve things further.

    Paul

  6. @ Paul

    Digging into the OPN files, I can’t tie up the Message2V4 class to the media type (E085) in the exported .cap file. Is there a mapping somewhere?

  7. Paul E Long says:

    Ed, you might be able to do some of this today with Message Analyzer. Assuming you can access both machines with our Remote Capture feature, you could collect a trace from both sides. Then, you could group by IP Identification and see any groups with one
    message, meaning it didn’t appear at both sides. Or a different approach would be to group by DataSource and Port, which should result in the same number of messages from both sides. A simple Telnet script that attaches to the ports you are interested in can
    be used to interrogate the firewall.

    Another indication of messages getting lost in general is the TCP Retransmit/Lost Segment diagnosis. Even from one side you can see which messages are not being delivered. By opening the Diagnosis tool, you can see a all the diagnosis messages which can help
    you drill in and understand if a firewall is blocking traffic.

  8. Paul E Long says:

    As for TLS/SSL Decryption, we’ll post more details in the help in next few days. But the short answer is, 1. Goto File->Options, and add the Private Cert and PW. 2. Open your trace. 3. Open the Decryption tool window. 4. Click on a line in the tool window
    to see the related decrypted data in the analysis grid.

    Keep in mind it’ snot perfect. Sometimes it doesn’t always detect incomplete TLS/SSL sessions, so the list in the tool doesn’t always account for every TLS/SSL session in a trace.

  9. Paul E Long says:

    @Mick, there is not an option. However you can still use the Network Monitor tool to configure the adapter, which was a completely different executable that configured the wireless adapter directly. Then you can still capture using Message Analyzer.

  10. Paul E Long says:

    There are a bunch of FiddlerCore properties that we have yet to expose. We certainly want to expose them, but I don’t have a timeline when the that might happen.

  11. Paul E Long says:

    Tmatt, can you right click and copy the shortcut into a comment? We could at least see if that matches.

  12. Paul E Long says:

    @Graham, the Firewall and HTTP Proxy providers are installed by MA. The NDIS one is inbox for 8.1 and forward. We are working on making the other providers inbox as well. I would follow up on the Forums if you have more questions, as it’s easier to track
    these types of conversations.

  13. Paul E Long says:

    @Graham, when you look at the traffic you captured, is the address 127.0.0.1 or ::1? If not, what does it show up as? Just curious to understand why our default scenario isn’t picking it up.

    I think it might be possible, but I don’t think we documented the means. They are just ETL providers, so any ETL consume can at least configure the provider in the natural way. However for WFP and HTTP Proxy providers, there are some other steps you have to
    take to configure and start/stop. If want more info I can try to get more details.

  14. Ed (DareDevil57) says:

    thanks

  15. Paul E Long says:

    @EricLaw, we are looking into the compression and limiting of the HTTP stream we use to gather our updated asset and news. Hopefully we can continue to refine this moving forward.

  16. Paul E Long says:

    @Graeme, WSUS is another feature we are looking to enable moving forward. Certainly understand the scenarios and of course helps us keep people up to date as well 🙂

    Paul

  17. Paul E Long says:

    Rafael, I’ve tried to fix some of the pictures, but that process never seems to work perfectly for me. I’ll try to investigate if there’s a better way to format them.

  18. Paul E Long says:

    @Jon, we’ll work more on discoverability after gathering some more feedback. I’m sure we can always improve things.

    I can check if we can self-elevate for this action. I agree that would be a better experience.

    For process tracking, it is on our radar. Also, there might be other solutions that get us more accuracy, like using the firewall driver instead of NDIS. We understand it was very useful.

    Paul

  19. Paul E Long says:

    Jason, we have not changed this behavior so my guess is that you’ll still have a problem with you whitelisting software. We might be able to address this, but can I ask if it requires Authenticode or Strong name signing?

  20. @Paul, thanks for the replies, if I need more about the providers I agree that the forums are likely to be a better place.

  21. Paul E Long says:

    @Jon, yes, we’ve discovered this issue as well. Funny thing was, as that I tried this on a airplane to verify I was giving you accurate steps. And I ran in to this very issue, determining that maybe you ran into this issue because we thought it should
    be fairly discoverable. We’ve since found repro steps, though it seems to be aggravated by a slow network 🙂

    Hopefully this is something we can address in the future and provide some information for others on the blog and start page.

    For Admin, this is true because we now use the inbox drive which decided to restrict access further. It’s not that Network Monitor self-elevated, but rather our driver model didn’t require it. Instead we used another security group, for the scenario where you
    don’t want to hand over the keys to the kingdom to get a trace. We are talking to the networking team, and have providing them our scenario. I can forward other feedback about this scenario if you have something specific to add.

    The closest equivalent for Network Conversations is the View Layout called Network Conversation Tree and Process ID. While we don’t capture processes like we used to, the system does a lose association where outgoing traffic is marked correctly. Incoming traffic,
    could be random. Since we can group by Request/Response, the result is that the process ID that we find first is the outgoing request. And so this view does a decent job of reporting the process ID. However, the process name/icon is not preserved with our
    scenarios. The NetSh WAN or LAN tracing does include the process names, which can be found in this view as one of the System level messages.
    If you need further details or help, feel free to ask in the forums, where it’s easy to discuss topics.

    Thanks,

    Paul

  22. Paul E Long says:

    Will, Process Tracking is something we are interested in adding. Since we moved to the inbox NDIS capture provider, we don’t own the driver anymore, and we are working with that team, but they are also have many different priorities to consider.

    For today, you can use the built-in Network Conversation with Process ID view layout to see traffic by Process ID, which is accurate for any traffic initiated by the capturing, though we realize this doesn’t list the process ID or Icon. Additionally, if you
    capture using the network scenarios Netsh, there are message that associate Process ID to the process name. In fact, I was able to create a quick chart by setting the Data Column=ImageFileName and the Data Row=ProcessId. Perhaps we can include this chart in
    a future update.

    Paul

  23. Paul E Long says:

    Norbert, you can get a log by using the command line and msiexec.

    Msiexec -package MessageAnalyzer64.msi -l* ma.log

    You can use the forums, which has better threading, to get more help. The link for the forums is on the right in the Resources section.

  24. @Paul, thanks for the doc info.

    The traffic is directed to "localhost.", and using the "Local Loopback Network" scenario failed to capture traffic. Using the "Loopback and Unencrypted IPSEC" scenario captured the traffic I mentioned. I’ve since found the configuration for the WFP Message
    provider that allows filtering on inbound or outbound, so selecting only one of those eliminates the duplicates.

    Bonus question, can any process hook into these providers, or is access restricted by some means?

  25. Paul,

    Is there any documentation available on new media types in the .cap files? I’ve captured IPv4 loopback traffic and it appears to have a media type of E085 that is some form of pseudo IP header. MA shows that part of the frame with a module of WFPCapture and
    a type(??) of Message 2V4.

    Also, with loopback captures I see the packet twice, presumably once on the transmission (with a TCP checksum of 0) and again on the receipt (with a valid TCP checksum), is there a way to filter out the dups?

  26. Paul E Long says:

    HB, the supported versions is listed in the System Requirements page. But I see if I can add some info to the blog to make this more obvious.

    Paul

  27. Paul E Long says:

    No, though it’s a great idea and something we have talked about for the future. For instance, we could tie back to the documentation so that when you look at SMB2 Command, you can see it’s description and automatically jump to the documentation on TechNet.

    If this is not what you meant, please describe further.

    Paul

  28. Ralph Case says:

    Thanks for your work on the new release. At first look, the performance does seem improved. I’m looking forward to trying out some of the new features.

  29. Paul E Long says:

    I suspect that it’s not .pcap that is the blocker, but instead that dissectors for wireshark are missing for certain types of data. Can you tell me what issues you are seeing? Better yet, create a new thread in our forums and we can investigate?

  30. @ Paul, the address is 127.0.0.1, the application I was testing with isn’t IPv6 capable.

    re the ETL providers, are they installed by MA, or are they already available in the OS? I might want to follow this up, thinking about a replacement for WinPCap.

  31. Paul E Long says:

    Tmatt, I just tested the link in the blog and it worked for me. Is that the link you are talking about? When I copy the shortcut, I see this:

    http://download.microsoft.com/download/2/8/3/283DE38A-5164-49DB-9883-9D1CC432174D/MessageAnalyzer64.msi. Is that what you get?

  32. Paul E Long says:

    @Graham, the media type is assigned by our export code. There are only certain things that make sense for .cap, so we handle those and hard code media types. Then created hooked up Network Monitor parsers to deal with them As for a list, let me see if
    we have something we can share.

  33. Paul E Long says:

    Kevin, I’ve updated the link and explicitly put the version number in the title. The version can be viewed from the File->About menu. There is of course also a build number, which matches the properties for the but this doesn’t match the version number
    because at the start of the project, we were thinking that we would release Network Monitor version 4.x. But this changed, and changing the build version number afterwards is difficult.

  34. Paul E Long says:

    Jon, to do a trace select File-> Quick Trace. Then select one of the scenarios listed. Local Network Interface captures like Network Monitor. You have other options as well with Message Analyzer, which is why it now takes 3 clicks :). BTW, the list that
    show up is based on your favorites which you can change from the New Session the selecting the Live Trace. Then use the Trace scenario drop down and click on the stars to add or remove scenarios as favorites.

  35. Ed (DareDevil57) says:

    thank you

  36. John says:

    Need details on how to use TLS/SSL Decryption

  37. Anonymous says:

    La version 1.1 de Message Analyser a été annoncée sur le blog suivant :
    http://blogs.technet.com/b/messageanalyzer

  38. Rafael says:

    Your screenshots look like garbage.

  39. From the known issues: "There’s no way to know the actual process ID or name of the traffic from the WebProxy provider." Why not? FiddlerCore exposes this information…

  40. Kevin Gould says:

    Looking forward to the new features. The sidebar link for "Message Analyzer Download" still points to the old one? It’s always irritated me that there was never a version or publish date indicated on that download. The new one also does not have a version
    number in the file. Why? When we save it, of course we can rename it to reflect what we view as the version number, but what if you have sub point-version releases – the numbering scheme for these should be set by MS to eliminate confusion.

  41. Opening the tool downloads a 550KB ATOM feed (http://maodatafeedsservice.cloudapp.net/MAODataAssets.svc/Assets). Can HTTP compression be enabled to shrink this to ~60kb?

  42. christoph says:

    Interesting, I know Network Monitor but I’m surprised this tool has been around for 2 years and I haven’t heard of it until now. Will give it a shot now even though I wasn’t a fan of Network Monitor..

  43. Albert says:

    Paul, thank you, and those others who are involved in this project, for your efforts in answering our questions and addressing our concerns. The new release looks great.

    ~Albert

  44. will says:

    Hi Paul, first of all, thanks for all the effort you’ve made in this wonderful tool.
    One thing that I really miss is, like in NM, to see what process is generating the traffic.

    Is this feature be added in a future reléase?

    Thanks!

  45. hb says:

    after download of "package-install" it tells me – u have VISTA- i don’t work on vista, sorry. that’s not nice, y not just tell me at the top of this page.

  46. Ed Woodrick says:

    One think that I was just thinking about that could often be a great help is a network analyzer. I’m talking a little different than the normal definition. The concept would be to be able to deploy two machines, each on either side of a firewall, and run
    port/application tests between the two. It could help to both test a firewall’s policies and to find out what it is exactly doing, as many of us consultants who go on to customer sites have to do.
    It would also be pretty nice if a reference system existed on the Internet, so that you could just run the test from inside the customer’s network and see what the result is for Internet connectivity.
    This would be really great for testing for Office 365 connectivity, hopefully much better than the marginal tools that exist today. Max throughput, port blockage, Lync jitter, all of these would be great information.

  47. Clay Shannon says:

    The scream shots look fine to me.

    "Keep in mind it’ snot perfect"

    Snot is never what I would call perfect.

  48. Tmatt says:

    FYI – your direct download links give "The signature of MessageAnalyzer64 is corrupt or invalid" (at least for the 64bit). Download Center works great though. Looking forward to using this, I’ve been needing a tool like this for some of our socket connections
    that keep having intermittent issues.

  49. Tmatt says:

    Thanks for looking at that Paul. I still get the issue so it’s probably something on my side.

  50. Jon says:

    In Network Monitor (netmon.exe), it’s 2 clicks to start collecting data: (1) New Capture (2) Start.
    How many clicks is it in Message Analyzer?

    (I spent about 2 minutes trying to figure that out, gave up, and went back to netmon).

  51. Norbert Holtkamp says:

    Hi, trying to install on a 64bit W7 Ultimate I got an error while "updating component registration": Schlüssel im angegeben Status nicht gültig .. and the Installation was ended. Is there an installtion-log to see the reason ?

  52. Jason Fare says:

    In the previous verison, the embeded DLLs that are extracted to the %temp% location were not signed so our application whitelisting software blocked them and we were unable to use the product. Was that corrected in this version?

  53. Serge Mera says:

    Great work! I took it for a spin and performance has indeed improved 🙂

  54. Piotr Siódmak says:

    Here’s a good tip: if you have known issues regarding upgrade like this one in your Known Issues document:

    "Existing OPN parser files are deleted during uninstall
    The %localappdata%MicrosoftMessageAnalyzer folder which includes OPN parser files installed by Message Analyzer will be deleted when you uninstall or upgrade Message Analyzer. If you have added files or made any changes to files in this folder, make sure
    you copy them to a different location before uninstalling or upgrading Message Analyzer."

    then please put that information at the beginning of your announcement blog post. Some people just jump in on the new version thinking that just installing it will not hurt, while actually it might make you lose your data.

    As for the product itself, I’m looking forward to more performance improvements. I once had a 10 minute trace made with "netsh trace start" and after adding some filters MA killed my machine. It would help if there was an offline (as in "don’t load the whole
    trace at once") way to cut a trace, say "take only the stuff from 12:13:04 to 12:13:06 and skip ARP packets" because I already correlated the time with other logs, so I don’t need the rest of that trace to hog my RAM.

    Cheers,
    Piotr Siódmak

  55. Jon says:

    @Paul, thanks for the response. A couple of issues with Quick trace: (1) if you select it too quickly after startup, nothing appears in the right drilldown (bug?) (2) You have to remember to run as admin (which costs a few more clicks), unlike netmon which
    elevates immediately. Shouldn’t message analyzer ask to elevate?

    Next, the results. Netmon has a "Network Conversations" which let you very quickly see and filter which processes you want. Where is the equivalent in Message Analyzer?

  56. Jon says:

    @Paul, you’re right that I originally clicked on "Quick Trace" and it did nothing, so I thought it wouldn’t help me. On discoverability, "Quick Trace" should be in the ribbon so that it’s more visible and saves a click.

    For Admin, I’m referring to this error: "You might need to run Message Analyzer as Administrator to perform a live Link Layer trace." Rather than telling me what I might need to do, offer to do it for me 🙂 Compare this with Process Explorer’s "Show Details
    for All Processes" which elevates itself rather than giving an error.

    Regarding "Network Conversation Tree and Process ID", you’re right that it’s not as good as netmon’s "Network Conversations". Is this something that will be improved in future versions?

  57. Mick Taylor says:

    Is there an option to set a wifi adapter to monitor mode like in Net Monitor?

  58. Jon says:

    Thanks Paul.
    It’s a little disappointing that the prior tool that worked so well (netmon) was abandoned rather than incrementally improved, and the shiny new tool doesn’t have the same features as the old. Hopefully there were very good reasons for this decision.

  59. Anonymous says:

    Micosoft Message Analyzer

  60. Pcap export says:

    Wireshark doesn’t support *.cap format very well. Do you have plans to export also to *.pcap? It would be very useful for mixed systems environment.

  61. thomas says:

    Paul,
    Is there a way to add a display that describes a particular protocol field based on information I provide?