Troubleshooting NIS was never made easier

Article
12/15/2009

In the blog post from last week, we announced the availability of the whitepaper for configuring, monitoring and troubleshooting NIS in Forefront TMG 2010. NIS helps protect users against network based attacks. This whitepaper includes both technical background as well as step-by-step guidance. In this blog post we would like to focus on the troubleshooting information which is included in the whitepaper.

Troubleshooting Scenarios

NIS is easy to configure and monitor. If you still run into configuration issues or other difficulties, the troubleshooting section provides helpful information. It specifically addresses the following issues:

1. Signature set updates failure

2. Potentially incorrect detection

3. Potentially incorrect protocol anomaly detection

4. Potentially missing detection

Signature Set Updates Failure

To keep your systems protected from the latest threats, NIS must have connectivity to the appropriate update source (Microsoft Update or WSUS) and be updated with latest signature sets. NIS will trigger an alert on any failure to update the latest signature set and will display a warning on the Forefront TMG management console Alerts tab. When the signatures are up to date, the Update Center appears as follows:

Forefront TMG Update Center

NIS will alert if it hasn’t received updates for more than a certain number of days. This number is configurable through the Intrusion Prevention System node, on NIS Tasks. Select Configure Properties and then select the Definition Updates tab. This threshold is 45 days by default:

Threshold for triggering a “NIS signatures out of date” alert

The whitepaper provides the list of possible NIS alerts for signature set updates along with the corresponding troubleshooting steps. Please refer to pages 47-50.

Potentially Incorrect Detections

In rare cases, NIS may incorrectly detect legitimate traffic as a potential threat and block it if configured to do so. The NIS team uses telemetry to constantly enhance the quality of protocol decoders. In particular, Microsoft Malware Protection Center (MMPC) uses telemetry to maintain high-quality signatures in order to minimize the impact of incorrect detections and quickly respond to such events. If an incorrect detection is found, the updates follow a roll-forward model. A revised signature is then created, tested, and published to supersede the previous one. This way, customers benefit from the protection against the emerging threats that are addressed by the newer signature set. For information about the MMPC response please see the “Understanding the Research and Response for NIS” section in the whitepaper.

If you suspect that a specific signature is causing incorrect detections, you should set the specific signature action to Detect only and report the issue to Forefront TMG Customer Support. Some information, such as a network capture file, may be requested as part of an investigation. If you joined the Microsoft Telemetry Service in Advanced Participation level, these reports will be helpful to the MMPC team in analyzing the incident and taking action as necessary. Please see the “Telemetry Service” section in the whitepaper for more information.

Potentially Incorrect Protocol Anomaly Detection

The Protocol Anomalies Detection feature can help identify malicious traffic on your network. However, there is some risk that protocol anomalies will trigger incorrect detections, especially in cases where applications were not implemented according to formal protocols specifications. When protocol anomaly detection occurs, both a log entry and an alert are added.

In the example below, anomaly detection was triggered because the GET request specified an invalid HTTP version of “1.2”. The log entry for this event is the same as other signature detection actions.

The Alerts tab shows the alert which was triggered following this detection:

NIS alert for a policy anomaly detection

Here is the specific alert text:

Protocol Anomaly Alert Text

When Telemetry Reporting is enabled, NIS will report the incident to MMPC to help identify and eliminate potential similar detections in the future.

When Forefront TMG alerts indicate that the session was blocked due to protocol anomaly, you may consider changing the Protocol Anomalies Policy to allow traffic instead of blocking it. To do that, follow these steps:

1. Select the Intrusion Prevention System node in the left pane

2. Select Configure Properties from the NIS Tasks menu.

3. Select the Protocol Anomalies Policy tab

4. Select the option Allow, to avoid blocking legitimate traffic.

Potentially Missing Detection

There are several potential reasons for lack of detection for an apparent exploit, sometimes referred to as a false negative. They are grouped as follows:

· File based exploits

· Signature policy configuration issues

· Network object exceptions

· Signature set version is not up-to-date

· User defined protocols

Please refer to pages 51-54 of the whitepaper for more details.