When troubleshooting various Exchange issues it can be very beneficial to get a network capture to look at the actual packets going over the wire. For example when looking at Outlook connectivity issues we can enable Outlook client logging and RPC Client Access Logging on the Exchange Server. Both are both great troubleshooting tools and while we can solve a lot of issues with that information, there can still be great value in looking at the actual network packets. Is there some inline WAN optimisation device that is causing an issue or do we see retransmits?
Unlike in days of old * we do not install the full version of NetMon from the SMS installation media, it can be downloaded from the Microsoft Download Centre.
Note that there are separate packages for x86, x64 and Itanium installations. Ensure that you chose the correct type else it will not install.
Installing Network Monitor is straightforward. It will install the tool itself, and then the parsers which are required to split and analyse the traffic. The parsers that come with the Netmon download are a bit out-dated nowadays. The latest parsers can be downloaded from Codeplex. Installing the newer parsers will overwrite the older ones.
By default Netmon 3.4 will install into C:\Program Files\Microsoft Netmon 3. This folder is used for both x86 and x64 installations since the image type is native to the OS.
I won’t go into detail in using the Netmon GUI, rather let’s focus on the command line aspects of capturing. As it can be easier to capture data this way rather than explaining what buttons or options someone has to select in the UI. From a Microsoft perspective this allows us to send out a command that we know will be correctly executed and the required data gathered. Else the data could be missed and we have to wait for another occurrence thus delaying the troubleshooting.
The GUI Network Monitor executable file is Netmon.exe. For command line work we need to use NMCap.exe.
Quick & Simple Examples
Some quick examples before we go down the rabbit hole……
Capture All Traffic To A Rolling Log File
NMCap.exe /Network * /Capture /File C:\Netmon.cap:100MB
Capture Traffic On A Named Interface – Local Area Connection
NMCap.exe /Network “Local Area Connection” /Capture /File C:\NetMon.cap:100MB
Capture All Traffic – Chained Log Files
NMCap.exe /Network * /Capture /File C:\Netmon.chn:100MB
Command Line Options
To explore all of the command line options open an elevated CMD prompt, and change directory to C:\Program Files\Microsoft Netmon 3 directory. Once there run
To see examples run:
I’ll let you pick through all of the syntax, but there are a couple to note. First up, the file extension controls if circular logging is used.
/File <Capture File>[:<File Size Limit>] Name of capture file to save frames to. Extensions are used to determine the behavior of NMCap.
.cap — Netmon 2 capture file
.chn — Series of Netmon 2 capture files: t.cap, t(1).cap, t(2).cap…
<File Size Limit> is optional. It limits the file size of each capture file generated. Default single capture file size limit is 20 MB. The upper bound of the file size limit is 500 MB. The lower bound of the file size limit depends on the frame size captured. (Note that the maximal size of Ethernet frames is 1500 bytes). The files are circular, so once the size limit is reached, new data overwrites older data.
Example Usage: /File t.cap:50M
There are various start and stop conditions that can be used. They can be time or action based. If noting is specified use Ctrl + C to stop the capture. Take a look at the help for more details on /TerminateWhen, /StopWhen and /StartWhen.
Capturing Data Via The Command Line
In Short the command line steps are:
Open elevated CMD prompt
Ensure C:\Netmon folder exists
Change directory to C:\Program Files\Microsoft Netmon 3 directory
Run the Netmon capture command
One thing to note! There are options to limit the networks that data will be captured on. For example Nmcap.exe /Network 3 could be used. How do we know what networks are present, and which interface is what?
NMCap.exe /DisplayNetwork to the rescue!
Netmon Bundled Examples
This example starts capturing all TCP frames and will be saved in a capture file name tcp.cap. If you want to stop capturing, Press Control+C.
nmcap /network * /capture tcp /File tcp.cap
This example starts capturing network frames that DO NOT contain ARPs, ICMP,NBtNs and BROWSER frames. If you want to stop capturing, Press Control+C.
nmcap /network * /capture (!ARP AND !ICMP AND !NBTNS AND !BROWSER) /File NoNoise.cap
This example starts capturing network frames that are TCP Continuations. The capture filter is searching for String “Continuation in TCP Frame Summary Description. In order to see the complete list of Netmon Properties that are filterable,type “.Property” in the Netmon Filter UI.
nmcap /network * /capture contains(.Property.Description, \”Continuation\”) /File TCPContinuations.cap
This example starts capturing network frames at 3:17 PM on September 10, 2002. All DNS frames that contains the QRecord Questions name ‘my_computer’ will be saved in a capture file named dns.cap. The size of the capture file will not exceed 6 megabytes. If the user presses x at any time during this capture, the program will terminate, otherwise the capture will stop 10 minutes after it has begun.
nmcap /network * /startwhen /time 3:17:00 PM 9/10/2002 /capture contains(dns.qrecord.questionname,’my_computer’) /file dns.cap:6M /stopwhen /timeafter 10Min /TerminateWhen /KeyPress x
This example starts capturing network frames after 10 seconds past. All IPv4 frames that received by local machine which has IP address 192.168.0.1 will be saved in a capture file named ip.cap. The size of the capture file will not exceed 4 megabytes (the default size). If the user presses c at any time during this capture, the program will terminate, otherwise the capture will stop 10 minutes after it has begun.
nmcap /network * /startwhen /timeafter 10 /capture ipv4.destinationaddress == 192.168.0.1 /file ip.cap /stopwhen /timeafter 10 min /TerminateWhen /KeyPress c
Starts capturing network frames immediately. All TCP frames that have a source port or destination port of 80 are saved to the chained capture files named test.cap, test(1).cap, test(2).cap, … When the user presses the ‘x’ key the program stops.
nmcap /network * /capture tcp.port == 80 /file c:\temp\test.chn:6M /stopwhen /keypress x
Starts capturing network frames immediately. All syn TCP frames that have the specified IPv4 network address are stored into to the capture file t.cap. The program stops when the TCP connections ends.
nmcap /network * /startwhen /frame tcp.flags.syn == TRUE AND ipv4.Address == 192.168.0.1 /capture /file t.cap:8M /stopwhen /frame (tcp.flags.fin == TRUE OR tcp.flags.reset == TRUE) AND ipv4.Address == 192.168.0.1
This example reassembles fragmented frames of capture.cap at all layers possible. The resultant capture file, Reassembled.cap will contain the Reassembled payloads alongwith the original unfragmented frames.
nmcap /inputcapture capture.cap /reassemblecapture /file Reassembled.cap
This example starts capturing frames and will be saved in a capture file name result.cap. If you want to stop capturing, Press Control+C. When the free disk space is less than 20% the total space of current disk, capture will stop as well.
nmcap /network * /capture /File result.cap /MinDiskQuotaPercentage 20
This example starts capturing frames and also tracks processes that generated network traffic. The resultant output file is ProcessTraffic.cap.
nmcap /network * /capture /File ProcessTraffic.cap /CaptureProcesses
The examples below are from real troubleshooting incidents. Note that %computername% is embedded into the output file so that we can easily identify which capture is which. Just like in 1994 and we have CV~1.doc, CV~2.doc and CV~3.doc on a floppy disk, it’s a real pain looking at a bunch of files called capture.cap. Yes we can place them in folders, but they often get messed up!
One of the neat things is that NMCAP uses the same filter syntax as Netmon.exe. That means you can tweak and develop the capture filter in the UI and then transpose it.
In the examples below we are using the Blob mechanism to determine the data that we want to capture. While this is not explicitly documented in the NMCap help content, it does fall within the [FrameFilter] section. As discussed on the NetMon team blog, to create a filter using the Blob, you need to know the offset and length of the pattern you are matching. Often, the simplest way to do this is open a trace you’ve taken from the network you are interested in, and click on the field in question. Then look in the hex details for that location and offset.
Below is an example of this. We have highlighted the IPv4 Destination Address field. Note in the Hex Details pane, the Frame Offset is then show as 30 with the Sel Bytes of 4.
For this example the filter would be Blob(FrameData, 30, 4) == 192.168.2.40
Neat, eh? I have to thank Curtis Houck for introducing this to me
While it is possible to use this syntax for filtering, this many not be the most optimal in a high volume traffic scenario. In such cases we can convert the IP address to HEX and use that in the filer. For the above example, let’s do that conversion to HEX, the staring filter is:
Blob(FrameData, 30, 4) == 192.168.2.40
I do this in two steps, firstly the IP octets are converted to HEX using the Windows calculator.
Blob(FrameData, 30, 4) == CO.A8.2.28
Then the periods are removed, and the HEX prefix of 0x is added
Blob(FrameData, 30, 4) == 0xCOA8228
Advanced Example 1
Capture all data between Source IP 192.168.16.5 and Destination IP 18.104.22.168.
In this example we will capture on any network, limiting FrameLength to 256 Bytes with an series of chained output files each of which are limited to 100 MB. The capture filter uses the Blob methodology described above for high performance parsing. Blob(FrameData,26,4)==192.168.16.5 is the filter for IPv4 Source Address. Blob(FrameData,30,4)==22.214.171.124 is the IPv4 Destination Address filter.
Nmcap /network * /maxframelength 256 /capture Blob(framedata,26,4)==192.168.16.5 or Blob(framedata,30,4)==126.96.36.199 /file C:\NetMon\%computername%.chn:100MB
Advanced Example 2
Capture all data between to Destination IP 192.168.2.15 on network interface 6
This example is similar to the
Nmcap /network 6 /maxframelength 256 /capture Blob(framedata,30,4)==192.168.2.15 /file C:\NetMon\Capture\%computername%.chn:100MB
* – I still think that I miss having to re-install the service pack on NT and getting prompted to restart just by looking at the network connection properties. I think….