Use PowerShell to maintain IIS logs
Summary: Learn how to use PowerShell to maintain and work with IIS logs.
Welcome back guest blogger, Terri Donahue. Here are Terri’s previous blog posts. Terri is a Senior Customer Support Engineer for Dynamicweb NA and a Visual Studio and Development Technologies MVP. Twitter: @terrid_dw
Internet Information Services (IIS) log maintenance has been a thorn in the side of web administrators for a while…basically since the first release of IIS. There isn’t a built-in utility to handle file compression, archival of log files, or deletion of log files. Depending on a site’s traffic, these logs can grow quite big very quickly and begin to cause disk space issues. Some available utilities compress and delete log files, but all utilities that I have seen and that also handle archival have been home grown. Not everyone has a business need for archiving log files. In my experience, there have been times that historical logs were needed to track down when a specific issue started occurring. You would be surprised how long it sometimes takes for issues on a website to be reported for troubleshooting. I put together a script that uses functions for file compression, archival, and deletion. By modularizing the script, it is easy to select only the functions that you want to use rather than having to use comment blocks to disable parts of the script.
There are a few things to decide before you implement this script.
- How many days of unzipped logs do you want to keep?
- How many days of non-archived zipped logs do you want to retain in the current folder?
- Where do you want to archive the older zipped logs – local or remote?
- How long do you want to retain the archived logs?
- Do you want to log what was done?
For the purposes of this script, I chose the following answers:
- Up to seven
- Up to seven
- Local archive directory
- Six months
- Yes
Global variables
Because I created the script as a group of functions, I used global variables rather than assigning file locations for each piece of the script.
#Setting Global variables
$Global:logDir = 'c:\\wwwlogs\\' #Location of IIS logs
$Global:archDir = 'c:\wwwlogs-archive\' #archive directory location
$Global:logFile = 'c:\temp\maintain-iislogs.txt'
$global:logTime = Get-Date -Format 'MM-dd-yyyy_hh-mm-ss'
Clear-Content $LogFile
I intentionally used \\ in the log directory variable because I am using it as the pattern for a -replace command in the archive portion of the script. This has to be done to escape the \ that is needed to be retained.
Log compression
I began by working on the function to compress existing log files. The script requires 7-Zip. It could be modified to use standard Windows compression if you prefer. If you don’t want to install 7-Zip on your server, you can simply copy 7z.exe and 7z.dll from another computer to the computer where you want to run the script. The value of the $days variable is a negative number because it determines how many days in the past you want to go. This variable is used with the Get-Date AddDays method. I also wanted to ensure that the zipped files maintained the original creation date rather than the date the file was zipped. I used the LastWriteTime method to set this value on the new zip file.
Function Compress-Logs
{
<#### 7-Zip variable I got it from the below link
#### http://mats.gardstad.se/matscodemix/2009/02/05/calling-7-zip-from-powershell/
# Alias for 7-zip ##>
if (-not (Test-Path -Path "$env:ProgramFiles\7-Zip\7z.exe")) #update path to point to the location of 7-zip
{
throw "$env:ProgramFiles\7-Zip\7z.exe needed"
}
Set-Alias -Name sz -Value "$env:ProgramFiles\7-Zip\7z.exe"
$days = ‘-6’ #this will result in 7 days of non-zipped log files
$logs = Get-ChildItem -Recurse -Path $logdir -Attributes !Directory -Filter *.log | Where-Object -FilterScript {
$_.LastWriteTime -lt (Get-Date).AddDays($days)
}
foreach ($log in $logs)
{
$name = $log.name #gets the filename
$directory = $log.DirectoryName #gets the directory name
$LastWriteTime = $log.LastWriteTime #gets the lastwritetime of the file
$zipfile = $name.Replace('.log','.7z') #creates the zipped filename
sz a -t7z "$directory\$zipfile" "$directory\$name" #runs 7-zip with the provided parameters – name and location of the zip file and the file to zip
if($LastExitCode -eq 0) #verifies the zip process was successful
{
Get-ChildItem $directory -Filter $zipfile | % {$_.LastWriteTime = $LastWriteTime} #sets the LastWriteTime of the zip file to match the original log file
Remove-Item -Path $directory\$name #deletes the original log file
$logtime + ': Created archive ' + $directory + '\' + $zipfile + '. Deleted original logfile: ' + $name | Out-File $logfile -Encoding UTF8 -Append #writes logfile entry
}
}
}
Zipped log archival
This function moves zipped logs that are older than the retention period from the original IIS logs folder location to the archive location that’s set in the global variables.
Function Archive-Logs
{
$archiveDays = '-13' #this will provide 7 days of zipped log files in the original directory - all others will be archived
$logFolders = Get-ChildItem -Path $logdir -Attributes Directory #gets the folders in the log directory
$zipLogs = Get-ChildItem -Recurse -Path $logdir -Attributes !Directory -Filter *.7z | Where-Object -FilterScript {
$_.LastWriteTime -lt (Get-Date).AddDays($archiveDays)
} #gets the zipped logs
foreach ($logFolder in $logFolders)
{
$folder = $logFolder.name #gets the directory the logfile is in
$folder = $folder -replace $logdir, '' #removes the original log directory keeping on the child portion of the name .ie c:\wwwlogs\w3svc becomes w3svc – needed for the folder creation and file move portions of this function
$targetDir = $archdir + $folder
if (!(Test-Path -Path $targetDir -PathType Container)) #checks if the folder exists in the archive location
{
New-Item -ItemType directory -Path $targetDir #creates folder if it doesn’t exist
}
}
foreach ($ziplog in $zipLogs)
{
$origZipDir = $ziplog.DirectoryName #gets the current folder name
$fileName = $ziplog.Name #gets the current zipped log name
$source = $origZipDir + '\' + $fileName #builds the source data
$destDir = $origZipDir -replace $logdir, '' #removes the parent log folder
$destination = $archdir + $destDir + '\' + $fileName #builds the destination data
Move-Item $source -Destination $destination #moves the file from the current location to the archive location
$logtime + ': Moved archive ' + $source + ' to ' + $destination | Out-File $logfile -Encoding UTF8 -Append #creates logfile entry
}
}
Zipped log deletion
And finally, the cleanup/delete function. The retention period differs depending on the application. For example, PCI solutions require a year or more of archived logs to be retained. This value should correspond to your retention policy.
Function Delete-Logs {
$delMonths = '-6' #retains 6 months of logs - adjust to meet your company's retention plan
$delLogs = Get-ChildItem -Recurse -Path $archdir -Attributes !Directory -Filter *.7z | Where-Object -FilterScript {
$_.LastWriteTime -lt (Get-Date).AddMonths($delMonths)
} #gets the list of logs older than specified for deletion
Foreach ($delLog in $delLogs) {
$filename = $delLog.Name #gets the filename
$delDir = $delLog.DirectoryName #gets the directory
$delFile = $delDir+ '\' + $filename #builds the delete data
Remove-Item $delFile #deletes the file
$logtime + ': Deleted archive ' + $delfile | Out-File $logfile -Encoding UTF8 -Append
} #creates the logfile entry
}
Calling the functions
The final piece of the script is to call the functions. You can remark out any of the functions that you don’t want to use.
Compress-Logs
Archive-Logs
Delete-Logs
The full script can be downloaded here on the Script Gallery.
Thanks, Terri, for sharing your time and knowledge.
I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. Also check out my Microsoft Operations Management Suite Blog. See you tomorrow. Until then, peace.
Ed Wilson Microsoft Scripting Guy
Light
Dark
0 comments