How To Instrument Catching Exceptions for Java APM

This post applies to both the Java APM Agent available from Application Insights as well as the RTM release of SCOM 2012 R2 (this feature was not available for the preview release of SCOM 2012 R2). This article is written by myself and my co-author: Cijo Thomas.

This post will discuss Exception Events generated by the Java APM Agent.  To illustrate this, let me briefly explain how instrumentation works, the limitations of this, and how to modify the configuration files to account for cases interesting to you.  In an ideal world, a method might look like:

public foo() throws Exception {
  // Do stuff
}

If foo() has been identified as a method to instrument (i.e. it is specified in pmonitor.config.xml), then at runtime the code is instrumented to look like:

public foo() throws Exception {
prefix()
  try {
// Do stuff
postfix()
}
catch(Exception e)
{
handleException()
}
}

And generally, this works.... except for when it does not. Sometimes various web frameworks (such as Struts) may have code that catches and maps exceptions. True, the web browser may display a message and an exception stack trace, but from a code point-of-view there was no exception. Consider this code:

public callRestService(){
  try {
    // contains calls to all other methods, which could throw any exceptions.
  } catch(Exception ex){
    logException(ex);
  }
}

In order to address this (common) software pattern, there are two suggestions:

  1. ExceptionHandler instrumentation
  2. Stackborder instrumentation

Using ExceptionHandler Instrumentation

What it that?  ExceptionHandler instrumentation is for methods which are called by user code(or a framework) when an exception is occurred. These methods typically do some logging or hide exception from surfacing to webpage. As the goal is to capture exception information, we will make the assumption that these methods take the exception object as one of the function's argument. When a method is instrumented as ExceptionHandler, this instrumentation extracts the exception object from the argument list and creates an exception event.

For example, consider the before mentioned code with a master try/catch statement.

public callRestService(){
try {
// contains calls to all other methods, which could throw any exceptions.
} catch(Exception ex){
logException(ex, "uh-oh!");
}
}

All exceptions thrown from user code gets caught in callRestService() and the method calls logException() to deal with the exception.  This would imply that in logException() is called, then there has been an exception.  If the method logException() is instrumented as ExceptionHandler, then when it is called the instrumentation code will generate an exception event.

Using this type of instrumentation is easy.   Just edit the ss:applicationSetting section of the pmonitor.config.xml file. All that needs to be done is to declare an exceptionHandler and declare the type of instrumentation to use.  A sample XML snippet is below for an exception logging method from a sample class called com.contoso.logger.logException (so if you substitute your package name and method in this syntax, you can just paste this into your configuration file).  Once done the logic describe above will be called with the exception handling code gets called.  An important caveat of note: this will only work if the method takes an exception object as one of its parameters.

The following shows the entries to be made in Pmonitor.config file to instrument the method as ExceptionHandler.

<ss:applicationSetting>
...
<ss:exceptionHandlers>
<ss:exceptionHandler name="com.contoso.logger.logException"/>
</ss:exceptionHandlers>
...
</ss:applicationSetting>
...
<ss:producers>
...
<ss:method name="com.contoso.logger.logException">
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.GenericExceptionHandlerProducers.GenericExceptionHandlerPrefixDataProducer " value="prefix"/>
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.GenericExceptionHandlerProducers.GenericExceptionHandlerPostfixDataProducer " value="postfix"/>
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.GenericExceptionHandlerProducers.GenericExceptionHandlerOnExceptionDataProducer " value="onException"/>
</ss:method>
...
</ss:producers>

Using StackBorder Instrumentation

What is that?  Suppose what is really important to know is that when a given method throws an exception - regardless of whether this event gets caught elsewhere in the code.

public callRestService(){
try {
...
canaryInTheCave()
...
} catch(Exception ex){
   logException(ex, "uh-oh!");
  }
}

public canaryInTheCave() throws Exception {
// some bellweather code
}

In this case, the method to instrument is canaryInTheCave() decides to throw an exception and the type of instrumentation is StackBorder. If any unhandled exceptions is thrown from a method with this instrumentation, it will result in "Critical Exception".  What this means is that irrespective of whether this exception is handled at a higher level or not, the code will create an exception event. This is ideal for scenarios where all exceptions generated from user code are handled by a framework code (or even a master try catch in use code itself). Without this type of instrumentation, this scenario would not generate any exception events because at a code level no exceptions surface.  Some other level of framework code is handling the exception and the StackBorder instrumentation comes to rescue. By instrumenting the method which stands at the boundary of user code and framework code (aka the stack border), then the exception crossing the method will be marked as critical and an exception event will be created.

This is not any more difficult from an XML perspective, an appropriate method should be marked in the pmonitor.config.xml.  The catch is to know the method name!  This is little tricky (we're looking at you Struts) as one must identify method(s) between where the user code generates an exception and the framework code that handles the exception. 

The following shows the entries to be made in Pmonitor.config file to instrument the method as StackBorder.

<ss:applicationSetting>
...
<ss:stackBorders>
<ss:stackBorder name="com.contoso.restService.canaryInTheCave"/>
</ss:stackBorders>
..
</ss:applicationSetting>
..

<ss:producers>
...
<ss:method name="com.contoso.restService.canaryInTheCave">
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.LightWeightProducers.LightWeightPrefixDataProducer" value="prefix"/>
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.LightWeightProducers.LightWeightPostfixDataProducer" value="postfix"/>
<ss:instrumentation name="com.microsoft.ManagementServices.APMAgent.Producers.LightWeightProducers.LightWeightOnExceptionDataProducer" value="onException"/>
</ss:method>
...
</ss:producers>