Introducing the Microsoft Exchange User Monitor (Exmon) tool


EDIT 9/23/2009: We have updated the download link to the Exmon tool which now also works on Exchange 2007 as well as Exchange 2010.

I am pleased to announce that this month’s Exchange Web release contains a tool that the Exchange performance, development, and operations teams at Microsoft have used for quite some time called Exchange User Monitor (Exmon) and can be downloaded here.  Exmon for the first time allows an Exchange administrator the ability to see in amazing detail the performance of an Exchange server.  Shown on a user by user basis, Exmon allows you to see how much CPU, latency, network traffic, and disk each user on an Exchange server consumes.  It can be run in almost realtime (minute by minute analysis) or over longer (multiple-hour) capture periods.  Exmon also ‘bubbles’ up data sent back to the Exchange server from Outlook 2003 and higher about the user’s actual experience, showing the actual RPC (network+server) latency and even the name of the process talking to the Exchange server (so you can see ActiveSync usage and other 3rd party MAPI applications).  The data Exmon exposes is the ‘raw’ data that many of the Exchange Performance counters use in calculating the running averages.

Internally, this tool was used to help understand the performance of Outlook 2003 and other MAPI applications during the development of Exchange Server 2003.  We use it to understand the broad impact of performance across a server, but also to troubleshoot specific performance problems with individual users.  The impact to the server being ‘traced’ is minimal, allowing it to be run on very large servers.

I’d love for you to download the tool, give it a whirl, and tell us what you think.  We’d love to see what use you can come up with for this data, problems you’re able to solve, and conclusions you’re able to make.

On a personal note, I recommend looking at a number of traces to ‘aquaint’ yourself with what’s normal.  Exmon data can be highly dynamic in range.  It’s best to use longer traces rather than smaller ones, because MAPI traffic can be very bursty, unless you’re trying to look at a very microscopic level.

Chris Mitchell

Comments (33)
  1. Nurv says:

    Where can we find more info about this? I am in the process of migrating from Exchange 5.5 to Exchange 2003, I would like to know if this will help me in my migration process. Maybe get a baseline of 5.5 before moving to 2003

  2. Xavier says:

    Hi Nurv,

    There is a good documentation installed with the tool. By default located at C:program filesExmonUsing_Exmon.doc

    Thank you again for this great blog.

    Xavier

  3. JF says:

    Great Tool! Thanks!

  4. Anonymous says:

    Microsoft has established a good pattern: they’ve been taking tools that they use internally, polishing them up, and releasing them as free tools through their web release (WR) program. This flow most recently brought us ExBPA 2.0, and now a…

  5. JF says:

    As I said before, great tool; however I installed and ran a couple times and had to kill it manually through task manager at one point. I wasn’t able to get it restarted. unknown Start Trace Error (183). thanks.

  6. Mike V says:

    Will Exmon show data for OWA users ?

    Thanks.

  7. Briam says:

    Great Tool. On most of my servers, this detail will allow us to trend usage and versions.

    I do have one E2k3 server that does not have the Exchange Provider listed for perfmon/logman…. and the counters are enabled (exctrlst) Any thoughts?

    Briam

  8. eufreka says:

    Am I actually required to install this to my production exchange server BEFORE I can review any documentation?

    I mean really, there are no install docs, system requirements nothing? People are willy-nilly installing this on their EXCHANGE SERVERS?

  9. Chris Mitchell [Microsoft] says:

    Here are some answeres to your questions:

    1. Exmon only shows MAPI traffic at this time, no OWA

    2. Exmon can only gather data on Exchange 2000 SP3 and higher.

    3. This tool can be instlled on any machine. The only reason it needs to be isntalled on an exchange server is to use the tool in ‘live’ mode to collect data. The document describes ways to collect the data using normal means.

  10. Anonymous says:

    Is this a cleaned up version of ExTop?

  11. Vermyndax says:

    Was this tool removed? I get a 404 when I try to go download it from the link in your post.

  12. Shane Pollard says:

    I installed and started collecting data directly with exmon. The application stopped responding and so I killed the exmon.exe process. Problem is, when I go to run it now, I get an error saying "Unknown StarTrace error (183)". When I look in the exmon directory, there is a .etl file that is growing so I am assuming that exmon is still running somewhere in memory and collecting data. How do I stop it so that I can start exmon again?

  13. Exchange says:

    Shane, any way to contact you on this? You can drop me your email through here:

    http://blogs.technet.com/exchange/archive/2004/12/10/279655.aspx

    Thanks.

  14. james says:

    I get the following error when trying to start exmon: "Unkown StartTrace error (112)" any ideas on how to fix this?

    Thanks,

    James

  15. Matt says:

    What kinda of application can you use to view the .etl files that the Exmon creates?

  16. Exchange says:

    James – Error 112 is "disk full on the tracing disk".

    Matt – you use EXMON to open thos files up. Meaning – on the command line, type Exmon <the_name_of_the_ETL_file> and hit Enter, it should open up.

  17. claudio says:

    Hi. The tool seems very interesting… however I’m unable to get what I need most at the moment: client versions. In an E2K SP3 shop, with various sorts of clients (pre-OL2K3, mostly) I consistently read "0.0.0" in the version column. Has this ever happened to you? What can be done to fix it? Thanks!

  18. Chris Mitchell [MSFT] says:

    Claudio,

    Thanks for your interest in Exmon. Unfortunately, E2K SP3 doesn’t have support for exposing the MAPI version with Exmon. We added that feature in E2K3. Exmon was an ‘evolving’ tool in that we used it and then incrementally added features that we found useful, client versions was one of those additions. The clients can be any version, but unfortunately, you’ll need E2k3.

    Chris Mitchell

    Software Design Engineer

    Exchange Performance Engineering Team

    This posting is provided "AS IS" with no warranties, and no rights confered.

  19. Roland Schmid says:

    I’m having the same issue as some of the other guys in regards to the etl file growing and I cannot remove it. Plus when I re-start exmon after it hung I get the "Unknown Start Trace Error (183)"

    I don’t want the fix to be restart the server… I just have too many users hanging off it.

  20. KJ says:

    Hi,

    Wonder if anybody has tried ExMon on Exchange 2003 SP1 ?

    I am trying to figure out how to read the results obtained. I noticed that

    for some of the users, the Max Server latency (result by User View) is

    higher than the Max Client Latency (result by ClientMon View).

    Eg.

    User A,

    Max Server Latency (ms): 6374

    Max Client Latency (ms) : 1078

    Shouldn’t the Max Client Latency be always higher than the Max Server

    Latency ?

    Also wonder if there are any baseline for comparison of the results

    generated from ExMon ? I am trying to see how is the user experience like

    compared to the norm. Have been getting feedback from my users that Outlook

    often "hang", getting the "Outlook is retrieving data from Exchange" server

    message.

    cheers

    KJ

  21. Chris Mitchell [MSFT] says:

    Roland and others getting error 183:

    Yes, we’re aware of this issue. If Exmon crashes (or is killed) while collecting data the trace will continue. However, we do have a limit of 512 MB at which point the tracing will stop. The reason for the 183 error is that Exmon tries to start a new trace (while the old one is still tracing) and it can’t because only 1 trace is allowed at a time. Waiting for the file to hit 512MB will fix the problem. Alternatively, you can use the tracelog.exe tool (in the Windows Reskit) to stop the Exmon trace by the following commmand

    tracelog -stop ‘Exchange Event Trace’. When you run it though, it’s important to make sure the Exmon.exe process is not running.

    The reason for the crash (that leaves us in this state) isn’t known right now, but I have anecdotal evidence that the repro is this:

    1. Start Exmon (which starts tracing)

    2. Immediately goto File->Save (when there’s no data displayed yet in Exmon).

    The File->Save is NOT meant to specifiy the location of where the trace files go. Instead it’s used to SAVE the contents of the current trace into a CSV format. To change the current tracing directory, you should run exmon from the directory you want to trace into…

    For example, if you want to trace to x:blah, then you should do the following:

    1. x:

    2. cd blah

    3. "c:program filesExchange User MonitorExmon.exe".

    This will trace the files to the correct location. You could alternatively use the Perfmon collection method or the tracelog.exe method as specified in the help file.

    Chris Mitchell

    Software Design Engineer

    Exchange Performance Engineering Team

    This posting is provided "AS IS" with no warranties, and no rights confered.

  22. Chris Mitchell [MSFT] says:

    KJ:

    The Max ClientLatency (ms) is reported by Outlook to the Exchange server and can sometimes have a delay on it. Under most circumstances, the report is sent back to the server soon after the event, but not always. To improve the reliability, I would trace for longer period of time (changing the Update Interval (min) on the toolbar to 5 or 10 minutes OR using Perfmon or Tracelog.exe as documented to collect longer traces).

    Chris Mitchell

    Software Design Engineer

    Exchange Performance Engineering Team

    This posting is provided "AS IS" with no warranties, and no rights confered.

  23. Michael B. Smith says:

    When double-clicking on exmon.exe, I get a dialog error, and exmon does not start:

    Unknown StartTrace error (183)

    The only place I can find this error mentioned is here on the Exchange blog, that someone else had the issue….

    The ETL log file is still growing.

    I know what caused the problem (I think) – but how do I fix it without rebooting my Exchange server?

    Cause:

    1) Run exmon on a T/S session

    2) Disconnect the session

    3) Reset the session

    4) Sign back in to a T/S session and try to start exmon.

    I can be contacted via michael at briworks dot com – although I’ll have to reboot soon to fix this if no one has any input.

    Thanks,

    Michael

  24. KJ says:

    Hi Chris

    Actually, the results were obtained via perfmon over an 8hr period.

    btw, do you know of any baseline results for comparision purposes ?

    cheers

    KJ

  25. Giorgio says:

    i want know how may message every user receive from the internal domain e how many from external domain. is it possible???

    thanks

  26. Josh says:

    I have Exchange2000 with Outlook 2003 clients – will Exmon work in this type of environment?

    thanks

  27. Richie says:

    How to fix the StartTrace Error (183) error:

    Load the Windows Source Kit or just the tracelog.exe utility. Once loaded run the following command: Tracelog -stop “Exchange Event Trace”

  28. Oli says:

    Hi Chris,

    do you remember about my question in the microsoft.public.exchange.tools newsgroup from 20.04.2005 Subject: "Question about new ExMon Tool" ?

    I posted on 25.04.05 a few new interessting things about my measuring.

    Now I have a found out, that there must be a problem with the user-profile of this account. Remember, there is a user, who has open a group schedule with 500 users.

    If I start Outlook with the /rpcdiag switch, I can observe that there is a lot of rpc requests. Each 30seconds there is a considerable increase (about +3000 rpc request).

    If I now logon on the same computer with a different account and open the same postbox with the same group schedule, I cannot observe the same considerable increase. Only about each 5 min. I can see an increase of about 200-300 rpc requests.

    So the questions is:

    Are there any parameter in the registry to control the rpc requests period ?

    best regards

    Oli

  29. Anonymous says:

    Based on the questions that we got on another post, it seemed appropriate to address the &quot;Requesting…

  30. Anonymous says:

    Based on the questions that we got on another post, it seemed appropriate to address the &quot;Requesting…

  31. Anonymous says:

    Based on the questions that we got on another post, it seemed appropriate to address the &quot;Requesting…

  32. Kanwaljeet Singh says:

    Gud tool,

    But i need to check the exact time when users are recieving the mails.

    right now i m using promodag

    any idea about other tool..

    REgards

    Kanwaljeet

  33. Anonymous says:

    As it was mentioned in the previous post on the subject, Exmon (Exchange User Monitor) tool can be very…

Comments are closed.