How to generate a message trace with more than 5000 lines in PowerShell


Applies to: Exchange Online, Exchange Online Protection.

 

This scenario is addressed to companies that generate high mail flows or use multiple applications to relay emails.

Many admins are at some point facing a situation where analyzing a complex transport scenario is needed, this being translated into generating lengthy message traces.

 

The purpose of this article is to help Office 365 administrators generate message trace CSVs that contain more than 5000 lines each.

 

 

Microsoft documented the limitation of the 5000 results in a message trace in the following TechNet article: https://technet.microsoft.com/en-us/library/jj200712(v=exchg.150).aspx

 

The following script is using the parameters: “-Page” and “-PageSize” to go around this 5000 limitation:

Page Optional System.Int32 The Page parameter specifies the page number of the results you want to view. Valid input for this parameter is an integer between 1 and 1000. The default value is 1.
PageSize Optional System.Int32 The PageSize parameter specifies the maximum number of entries per page. Valid input for this parameter is an integer between 1 and 5000. The default value is 1000.

https://technet.microsoft.com/en-us/library/jj200704(v=exchg.160).aspx

 

By declaring the -PageSize to the maximum allowed value of 5000 and using the parameter -Page we force the trace generation to move to another page, incrementally, once the 5000 lines are filled with data.

 

DISCLAIMER: This application is a sample application. The sample is provided "as is" without warranty of any kind. Microsoft further disclaims all implied warranties including without limitation any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the samples remains with you. in no event, shall Microsoft or its suppliers be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss arising out of the use of or inability to use the samples, even if Microsoft has been advised of the possibility of such damages. Because some states do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you.

 

Before running the below script, you must meet the following prerequisites:

Global Admin permissions on Exchange Online.

PowerShell access to the Office 365 tenant.

 

 

 

$cred = Get-Credential

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $cred -Authentication Basic -AllowRedirection

Import-PSSession $Session

 

Start-Transcript -Path C:\temp\Transcript.txt -Force

 

$index = 1

while ($index -le 1001)

{

Get-MessageTrace -StartDate 08/28/2017 -EndDate 09/1/2017 -PageSize 5000 -Page $index | export-csv c:\temp\test.csv -Append

$index ++

sleep 5

}

Stop-Transcript

 

 

The same logic applies to Get-MessageTraceDetailed : https://technet.microsoft.com/en-us/library/jj200681(v=exchg.160).aspx

In addition to this sample transcript you can use all the parameters accepted by the Get-MessageTrace Get-MessageTraceDetailed cmdlets.

Comments (6)

  1. Alexandru Margoi says:

    Cool stuff (Y)

    1. Hope you find it useful 🙂

  2. Upgraded script version, forcing the loop to end once the message trace output stops writing in the CSV:

    $cred = Get-Credential

    $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $cred -Authentication Basic -AllowRedirection

    Import-PSSession $Session



    $index = 1

    do
    {

    $a = Get-MessageTrace -StartDate 11/14/2017 -EndDate 12/14/2017 -PageSize 5000 -Page $index
    $a | export-csv c:\temp\test.csv -Append

    $index ++

    }
    while ($index -le 1000 -and $a.count)

    Hope it helps 🙂

    1. Kate Paulin says:

      thank you for the update to the script, it was just what I was looking for- works perfectly

  3. __David_N says:

    Hi, how can I do this for x1 email address?

  4. I renew the script that get page until just last page.
    See below.

    # GetMailLog.ps1
    # Composed by T.Yamada 2018
    #
    # This Powershell script can get Microsoft Office365 Exchange online,
    # Mail with attachment transfer log.
    # You should configure that if attachment exist then add string to mail subject,
    # in your ExO transport rule.

    #
    $User = “admin@cepq.work” # User name for connect Office365
    $PasswordFile = “C:\Users\hato\password.txt” # File path of user’s password
    $OutputCSVFile = “C:\Users\hato\MailSendLog.csv” # Output log file path

    [System.String]$TargetDomain = “cepq.work” # Mail from Domain that you will get log (ex: “hato.co.jp”)
    [System.String]$FilterSubjectPhrase = “FileAttach” # String in mail subject configured on transport rule

    if (Test-Path -Path $OutputCSVFile) {
    Remove-Item -Path $OutputCSVFile
    Write-Host (get-date) “Remove old log file.”
    }

    #
    Write-Host (get-date) “Check Session”
    $CurrentExOSession = Get-PSSession -ErrorAction Stop| `
    Where-Object{$_.ConfigurationName -eq “Microsoft.Exchange” -and `
    $_.ComputerName -eq “outlook.office365.com” -and `
    $_.State -eq “Opened” `
    }

    #
    if ($CurrentExOSession -eq $null) {

    $SecurePassword = Get-Content $PasswordFile | ConvertTo-SecureString
    $UserCredential = New-Object System.Management.Automation.PSCredential $User, $SecurePassword

    Write-Host (get-date) “Connect-PSSession”

    $RemoteExOSession = New-PSSession `
    -ConfigurationName Microsoft.Exchange `
    -ConnectionUri https://outlook.office365.com/powershell-liveid/ `
    -Credential $UserCredential `
    -Authentication Basic `
    -AllowRedirection `
    -ErrorAction Stop `
    -WarningAction SilentlyContinue `
    -InformationAction SilentlyContinue

    Write-Host (get-date) “Import-PSSession”
    Import-PSSession -AllowClobber -ErrorAction Stop -WarningAction SilentlyContinue -InformationAction SilentlyContinue $RemoteExOSession
    }

    # <Log compose Definision>
    [System.Int32]$Page = 1 # Page Number start from "1"
    [System.Int32]$PageSize = 5000 # Number of log line in Page. Max value is 5000
    [System.Boolean]$haveNextPage = $true #Exist next page or NOT

    #
    while($haveNextPage) {

    Write-Host (get-date) “Processing in Page” $Page”…”

    $RawLog = $null # raw log
    $JSTLog = $null # modifiled log (single line)
    $JSTLogs = @() # modified log (multi line of this page)

    #
    $RawLog = Get-MessageTrace `
    -StartDate (Get-Date -hour “0” -minute “0” -second “0”).AddDays(-1) `
    -EndDate (Get-Date -hour “0” -minute “0” -second “0”).AddSeconds(-1) `
    -Page $Page `
    -PageSize $PageSize `
    -SenderAddress *@$TargetDomain `
    -Status Delivered `
    -ErrorAction Stop `
    -WarningAction SilentlyContinue `

    Write-Host (get-date) “Getting Log in Page” $Page”…”

    #
    $RawLog|ForEach-Object {

    if ($_.Subject.Contains($FilterSubjectPhrase)) {
    $JSTLog = New-Object PSObject|`
    Select-Object ReceivedDateTimeJST, ` # mail send date in japan standard time
    SenderAddress, ` # Mail address of sender
    RecipientAddress, ` # Mail address of recipient
    Subject, ` # mail subject
    Size # mail size (kilo-byte)

    $JSTLog.ReceivedDateTimeJST = (($_.Received).AddHours(9)).ToString(“yyyy/MM/dd HH:mm:ss”)
    $JSTLog.SenderAddress = $_.SenderAddress
    $JSTLog.RecipientAddress = $_.RecipientAddress
    $JSTLog.Subject = $_.Subject
    $JSTLog.Size = [math]::round(($_.Size)/1024,0)

    $JSTLogs += $JSTLog
    }
    }

    Write-Host (get-date) “Converting each Log line in Page” $Page”…”

    # <export log as csv>
    try {
    $JSTLogs|Export-Csv -Path $OutputCSVFile -Force -Encoding UTF8 -Append -NoTypeInformation -ErrorAction Stop
    } catch {Write-Host $_.exception}

    if ($RawLog.Count -eq $PageSize) {$Page++} else {$haveNextPage = $false}
    }

    #
    Remove-PSSession $RemoteExOSession

Skip to main content