A customer recently told us as he was experiencing high CPU in two of Azure virtual machines that were operating as web servers. The process that was consuming the CPU was monitoringhost.exe, which is a child process of the Microsoft Monitoring Host service, our OpsMgr and Log Analytics Agent.
The job of this process is to do all the monitoring and data collection asked of it by the configuration of OpsMgr or Azure Log Analytics. I looked at what the process was up to and could see that it was busy running the following function
So, we were busy uploading a file to the workspace and calculating a hash on them in doing this.
Checking the process further. I could see we were uploading the following file.
C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Monitoring Host Temporary Files 319\45\W3SVC1-u_extend1.log.iislog
We were uploading an IIS log as you can easily ask Log Analytics to do for you.
We checked these IIS log files on the affected servers and found that they were 14 and 15 GB in size!
They had been configured to never rollover as in the following screenshot.
Changing these to rollover on a schedule managed the size of the log files and returned CPU usage to normal levels.
So, the moral of the story is to be careful when you enable additional data to be sent to OMS/Log Analytics. Take some time to verify what you are asking the service to do, and what data is due to be uploaded.
Senior Escalation Engineer