The goal of this post is to show how DebugDiag 1.2 can assist you identifying a potential source of bottleneck on a scenario where TMG user mode process (wspsrv.exe) is consuming high amount of CPU.
First part is to make sure you collect the user mode dump while the issue is happening. To do that, use the approach that I explain in the following post:
Once you have the data you can use DebugDiag to analyze the dump. Follow the steps below in order to perform this analysis:
1. After installing Debug Diag (64 bits edition in this case), launch it and cancel the first window.
2. Click Advanced Analysis tab.
3. Click Add Data Files button and choose the dump file that was previously collected.
4. Choose the scenario that applies to this issue in the top pane. In this case the scenario is Crash/Hang Analyzers as shown below:
5. Click Start Analysis.
6. Wait until the report is generated.
Reviewing the Report
Don’t go too far on the report before reviewing the first part of it, which is the Analysis Summary. Here it is the example for this scenario:
In this case the warning message says:
Detected a possible critical section related problem in wspsrv.dmp
Lock at 0x015e7c70 is Unlocked
0.67% of threads blocked
The following functions are involved in the root cause
The thread number has a hyperlink on it, when you click on this hyperlink you will see the stack that it is referring to:
The recommendation that DebugDiag gives is:
The following vendors were identified for follow up based on root cause analysis
Unknown vendor for module C:\Program Files\Microsoft Forefront Threat Management Gateway\IPS\GapaEngine_1cc44e8_bace5e90.dll
Please follow up with the vendors identified above
In other words, it is telling me to investigate further this module. Now what? Well, now you have an initial path to follow, you know that GAPA Engine is involved, which means that you can start doing some tests, such as:
- Does the issue still happening if you disable NIS?
- If it does not, than you might be experiencing the issue described in this post http://blogs.technet.com/b/yuridiogenes/archive/2011/03/20/another-case-of-high-cpu-utilization-by-wspsrv-exe-on-forefront-tmg-2010.aspx
- If it does not, than you can try to narrow it down more by using the procedures from Forefront TMG 2010 Performance issues Cheat Sheet.
It is important to remember that troubleshooting performance issue can be a long process and DebugDiag can assist you to find the root cause. However, sometimes finding the culprit doesn’t fix the issue, just show who is causing the problem, in this case further investigation is needed to find out how to really fix the issue.