How to trace UAG in real time with filtering

Collecting a UAG trace is a common task for advanced troubleshooting. One of the challenges with regular tracing is the fact that it can produce almost a million lines of text per second, making it hard to find the relevant data in a trace. With busy servers, it’s also impossible to trace a server for more than a few minutes, as it will create huge trace files that can fill up the entire hard drive if not configured carefully.

One trick that can alleviate these in some situation is real-time tracing. The UAG trace tool has this option (the GoRT button), but then again, it’s clearly impossible to read all that output live. However, with a little bit of scripting, it can be used to monitor certain activities from the trace, filter the output, and even act on it (like, send an Email when something happens). To do this, we can output the real-time trace into a special output that a script can process. Then, we can perform any regular string-handling on this output and this way, we can parse, filter and manipulate it. Here are the steps to perform this sort of thing:

1. Download and extract the TMF files

TMF files are used to decode the function calls within the UAG code to human-readable text. Most of it still means little to non-programmers, but if you reached this article, you probably know your way around a UAG trace. The TMF files are available to download from here. Once downloaded, extract them to a folder on the UAG server, so that it contains them directly (several thousand files with GUID names). For example, the files could be in c:\TMFs

2. Configure the server to run VBScripts in a CMD window

We will be using a script to do this, but the Windows operating system is normally configured to run a script in a graphic environment (WSCRIPT), which is not suitable for our purposes. To change it, run the following command in an Administrator CMD window, to switch the default to the CMD version of the script engine (CSCRIPT):

Cscript //H:cscript

3. Create a local script to perform the tracing

In some folder, create the following file, and save it with the extension .VBS

This script uses the Wscript.shell object to launch the UAG real-time trace formatting utility TRACEFMT. The command is located in the UAG folder, so the script points to the full path for it. It also tells the command to run without a summary, and to decode the UAG provider only. In yellow, you can see it pointing to the folder with the TMFs (edit that path if you put them elsewhere than c:\TMFs).

Next, the script monitors the output of the process (oExec.StdOut) in a loop and reads it with the ReadLine method. Each line is put into the variable sLine, and then, we can check that variable’s content. The above example simply checks it, and is set to show only lines matching that specific text, which show the URLs that are being requested of UAG. If you are looking for a different activity, simple edit the green line and set whatever phrase you’re looking to see in real time.

4. Run the script, or set it do other stuff.

To run the script, you need to first start the trace with the UAG tracing utility. Launch it (<UAG Path>\common\bin\tracing\trace.hta) and set which components you want to track. Then, click GoRT, which opens a real-time trace without filtering.

Now, close that window, and in a CMD, run your own script, which will show only the filtered lines you need.

With the things UAG is doing being pumped into the sLine variable in real time, you can configure the script to do anything you want. For example, if you want to catch a 500 error that happens very randomly and rarely, you can look for the keyword “transmit500” or the key-phrase “500 internal server error”, and once they occur, perform some kind of action. Here are some examples:

The above function beeps 5 times if UAG runs into any action involving the UAG error page (Clearly, this filter may be too generic, but you get the idea, right?). Instead of a beep, you could have the script do whatever may be relevant – restart the IIS service:

Perform a STOP of UAG’s tracing:

Send an Email:

And, as usual, the possibilities are endless. You do need to have a good understanding of the way a UAG trace is built, so you can figure out what it is you’re looking for in the trace, but once you do, anything is possible.