IIS Log Format Requirements in Azure Operational Insights

[NOTE -  Operational Insights is now a part of Operations Management Suite. Learn more at microsoft.com/OMS ]

With regards to collection of ‘IIS Logs’ in Microsoft Azure Operational Insights, the only IIS Log format supported at the moment is W3C. Don't worry – it's the most common format, and the default one on IIS 7 and IIS 8.

But if you log in NCSA or IIS native format, we won't pick those logs up at all.

Even in W3C format, you must notice that not all fields are logged by default. Please read more about this log format in this article on TechNet.

For the best search experience, we recommend enabling all fields for each website as shown in the screenshot below:

IIS Logging Configuration for System Center Advisor Log Management


‘Computer’ field in Search

When enabling the s-computername field above, this gets mapped to 'Computer' field in our search index. Unfortunately, IIS by default logs the NETBIOS name of the computer. The other types of data produced by OpsMgr normally has computername in the FQDN format: COMPUTER.domain.com. This will lead to seeing 'duplicate' entries for computers in search, when using the measure command. This is being tracked here http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6772198-i-have-multiple-directly-connected-servers-listed and will be fixed by the upcoming change described here http://blogs.technet.com/b/momteam/archive/2015/05/14/configuration-changes-for-iis-log-collection-in-operations-management-suite.aspx 

 


Log File Rollover

IMPORTANT: We also recommend changing the rollover policy for new logs to 'Hourly' – so smaller files will be uploaded to the cloud, saving bandwidth.
Also, if you don’t change this, your management server might queue up the same files over and over again and we have had reports where it eventually runs out of space if the rate of incoming large files is higher than how fast your machines are able to save them to Azure Storage. 
This later issue with the OpsMgr attach topology is being fixed by the change described here http://blogs.technet.com/b/momteam/archive/2015/05/14/configuration-changes-for-iis-log-collection-in-operations-management-suite.aspx .

 

Custom Fields and other IIS-related logs

If you have additional custom fields that you add, we don't currently support those. There are some 1-off ideas for that http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6519267-collect-iis-advanced-logs and for the HTTPERR log http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6519313-collect-httperr-logs-in-addition-to-iis-logs

And if the site is running on Azure PaaS, check these other two ideas http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6519377-collect-iis-logs-from-windows-azure-diagnostics-st and http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6519351-collect-iis-logs-from-windows-azure-diagnostics-st

Anyhow, we are trying to work on the ‘generic’ platform capability to let you define your own log schema and fields. First step here http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/6519270-support-regular-expressions-regex-or-xpath-to-pe will be followed by the ‘collection’ pieces such as http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/7113030-collect-text-log-files and http://feedback.azure.com/forums/267889-azure-operational-insights/suggestions/7928931-collect-data-from-custom-containers-in-storage-acc

 


How to search IIS logs

Look at my list of sample searches http://blogs.msdn.com/b/dmuscett/archive/2014/10/19/advisor-searches-collection.aspx (find the ‘IIS’ section) and you might also want to read this other post with a couple sample search scenario around IIS logs http://blogs.msdn.com/b/dmuscett/archive/2014/09/20/w3c-iis-logs-search-in-system-center-advisor-limited-preview.aspx