Message Trace, the PowerShell Way


From my experience, a very small number of people actually choose PowerShell over the GUI (Graphical User Interface, ie. The Office 365 Portal). But once you get a grasp of PowerShell and write some scripts, you’ll see the light and going back to your old ways will be very hard. PowerShell has two big advantages. First, once you are proficient you’ll be able to pull and modify data much faster with PowerShell than going into the portal. Second, most managers aren’t PowerShell experts, so when you are working and have your PowerShell window open they will be extra impressed with you! Don’t worry, you can thank me later.

In this article I’m going to focus on using PowerShell to obtain message trace results from the past 48 hours. The commands in this article will work for date ranges up to seven days in the past. For messages older than seven days we need to run an extended message trace, or Start-HistoricalSearch.

Typical message trace run

Let’s start by running a typical message trace in EOP which returns all results for the past 48 hours.


 
In this view we can sort based on the columns presented to us and can double click for additional information. Now let’s look at how PowerShell can improve upon this experience.

Message trace using PowerShell

You’ll see below that I’m going to be piping my results to Out-GridView. If you have not used this cmdlet before, it essentially will launch a new interactive window where you can view the results. Let’s take a look.

The following script will return the message trace results of the last 48 hours.

$dateEnd = get-date
$dateStart = $dateEnd.AddHours(-48)

Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageID, MessageTraceID | Out-GridView 

Here is what the results look like.


 
Just like with the portal message trace, we can sort by clicking the column headers. Moving onto the differences, the first difference you’ll notice here is that there are more columns presented to us, and even nicer is that we can sort and filter on these extra columns which is something you can’t do in the portal message trace. For example, in the above screenshot, I have set a filter to only show me the messages that were larger than 12 MB, something that is not possible with the portal based message trace. Yes… now we are seeing some of the power!

 

Next let’s look for messages from the past 48 hours that have a Status of “Failed.” Here is our script.

$dateEnd = get-date
$dateStart = $dateEnd.AddHours(-48)

Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageID, MessageTraceID | Where {$_.Status -eq "Failed"} | Out-GridView

This shows that we have five failed messages from the last 48 hours. Now let’s pipe our results to Get-MessageTraceDetail to find out more information on the failed messages. Here’s our next script to do this.

$dateEnd = get-date
$dateStart = $dateEnd.AddHours(-48)

Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Where {$_.Status -eq "Failed"} | Get-MessageTraceDetail | Select-Object MessageID, Date, Event, Action, Detail, Data | Out-GridView
 

And this will return the following:

Here I can see the status of those failed messages. “Hop count exceeded – possible mail loop.” In this situation, a message was destined for a mailbox that didn’t exist in the cloud, because of a connector problem the message kept looping around Exchange Online before it was finally rejected.

Also notice the Data column above. Here we can see a lot of extra information that is not present in the portal message trace.

<root><MEP Name="ConnectorId" String="To_DefaultOpportunisticTLS"/><MEP Name="DeliveryPriority" String="Normal"/><MEP Name="OutboundProxyTargetIPAddress" String="207.46.XXX.XXX"/><MEP Name="OutboundProxyTargetHostName" String="contoso.mail.protection.outlook.com"/><MEP Name="RecipientStatus" String="[{LRT=};{LED=554 5.4.6 Hop count exceeded - possible mail loop};{FQDN=};{IP=}]"/></root>

What’s highlighted in yellow is all the portal message trace will show. The rest of the data can only be obtained with PowerShell, or through an Extended Message Trace. We can see the IP that issued the 500 error, the connector that was used, and even the MX record where Exchange Online was trying to deliver the message to.

 

Let’s take a look at another example. This time we want to track a particular message based on its Message ID. An end user told us that a sent message resulted in an NDR so we already know it failed. The NDR provides us with the MessageID. Let’s run the following to get directly to the message trace details for this particular message.

Get-MessageTrace -MessageId c857c16bf46d4586a198ec089a710706@BN1AFFO11FD033.protection.gbl | Get-MessageTraceDetail | Select  MessageID, Date, Event, Action, Detail, Data | Out-GridView


Right away we can see the error “Unable to relay.” We can also see this message was tried to be delivered using TLS. What we are seeing here is most likely a connector problem.

Of course, you don’t have to use Out-GridView, I just find it sometimes handy for quick filtering of the results.

 

Lastly, the searches I have provided above are fairly open ended. The tenant I used has a very low number of message coming through it. If you are finding that Get-MessageTrace is taking too long to run or timing out, I would suggest making your search stricter. For example, here is how you would search for messages coming from joe@contoso.com between a particular time range.

Get-MessageTrace -SenderAddress john@contoso.com -StartDate “12/15/2014 21:00” -EndDate “12/15/2014 22:00”

Note that the time specified must be in UTC.

Other Benefits

PowerShell can be set to run automatically. One example would be to create a script which checks for messages with a status of “Failed.” Have a server run this script every hour and if you see a large number of results, have the script or the Scheduled Task send you an alert.

Summary

For those that have never used PowerShell before it can look a little scary. There are a lot of great tutorials online to get you started and once you get your feet wet you’ll start moving fast. Script once, run many times.

On the personal side, I have a PowerShell script to automatically log me into my tenant. Once connected, I have scripts that pull Transport Rules, Message Trace results, Connector information, and much more. This is often faster than logging directly into the portal.

Finally, I'd like to thank Brian, one of my customers, who gave me the idea for this post.

Happy scripting!



Resources

Connect to Exchange Online using remote PowerShell
Exchange Online cmdlets

Connect to Exchange Online Protection using remote PowerShell
Exchange Online Protection cmdlets

Run a Message Trace and View Results
Get-MessageTrace cmdlet
Get-MessageTraceDetail cmdlet

Comments (34)

  1. HI Brian, it would definitely be handy to serach by subject in the GUI. Luckily we can do this with PowerShell though 🙂

    Try this to find all messages with a particular subject in the last 24 hours.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddHours(-24)

    $count = Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Where {$_.Subject -eq "Phishing attempt"}

    $count.Count

  2. I posted some content over the weekend and I thought I was logged in, but may not have been. Posts were to be moderated it looks like.

    I am really hoping you will approve those posts as I am sure the scripts and info will be appreciated by others.

    While I realize this is a MessageTrace, I was wondering if you might you have a lead on how to download the Get-HistoricalSearch FileUrl files from within powershell. I have been playing with Invoke-RestMethod and System.Net.WebClient to no avail.

  3. Hi Daniel, sorry for being very behind on my comment approving. I just approved the comments you wrote with your PowerShell scripts. Thank you for those!

    UCWARRIOR, not that it isn’t possible, but I’m currently not familiar with a way to search on IRM encrypted messages. If you had a rule that triggered the encryption, you could search based on that rule triggered. See
    https://technet.microsoft.com/en-us/library/jj200680(v=exchg.150).aspx.

  4. Hi Jeff, you are correct so I’ve updated the intro 🙂

    If you would like to run a trace that extends back 7 days, you could use the following in the above examples.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddDays(-7)

  5. Hi there Joe, unfortunately not. Message trace data is only available for the past 90 days. See
    https://technet.microsoft.com/en-us/library/jj200712(v=exchg.150).aspx.

  6. Brian says:

    Thanks Andrew, one feature that would be really great is if you could search on subject! IE how many messages did we get that were based on this phishing attempt. We often get asked this question

  7. pow er shell!…Pow er Shell!…POW ER SHELL!

    I will have to do this over a few posts I see (3072 char limit…ok fine).

    You know, you tease horribly with that little line, which I am sure many would love to see. You said "I have a PowerShell script to automatically log me into my tenant." NOW what kind of a monster are you for posting that personal triumph without helping your
    readers out!? 🙂 While I knock out my original comment I will contemplate posting how I do this…maybe some others will comment with a better way??? or more detail???

    All kidding aside, I do something very similar, but you have to remember, if you get a little too gung ho with your scripts Microsoft has limits that will throttle your connection.
    https://www.cogmotive.com/blog/office-365-tips/office-365-exchange-online-powershell-throttling
    http://blogs.msdn.com/b/exchangedev/archive/2011/06/23/exchange-online-throttling-and-limits-faq.aspx

    I run this little bad boy every night at 9:00 PM – I need to maintain this data in a SQL database for quick reporting of how many this or that, who sent how many to here or there, I think you get the point. It runs automatically via task scheduler.

    I get all the trace data for the last 24 hours, the only thing I cannot figure out for sure is if the output time's time is in what time zone – I am assuming it is UTC since that is what you see as default in the EOP. Something to note here is how I bypass
    the apparent 1000 record limit using the page parameter – I stumbled into it on accident (hooray for Get-Help!)

  8. # Gather a day's message tracking data for entire organization. Intended for a single day's data.
    # 8 day max offset as of 6/30/2014 – empty result sets if > 8
    # Script only requires that the variable $ReportDayOffset be set to the desired value.
    # I need to work out one issue where if the result set is empty an error is thrown.

    # 0 = "today so far", 1 = "yesterday", and 2 through 8 for up to 8 days back maximum.
    $ReportDayOffset = 1

    # Build value for -StartDate param's m/d/yyyy value:
    # 1) Set $StartDate = "Now" in short date format
    # 2) Subtract $ReportDayOffset "days" from $StartDate. Result is in long format
    # 3) Re-format $StartDate to short date format
    $StartDate = (Get-Date -Format d)
    $StartDate = (Get-Date $StartDate).AddDays(-$ReportDayOffset)
    $StartDate = (Get-Date $StartDate -Format d)

    # Build value for -EndDate param's m/d/yyyy value:
    # 4) Set $EndDate = $StartDate plus 1 "day". Result is in long format
    # 5) Re-format $EndDate to short date format
    $EndDate = (Get-Date $StartDate).AddDays(1) # StartDate var date +1 day (result is in wrong format)
    $EndDate = (Get-Date $EndDate -Format d) # re-formatted to m/d/yyyy

    # Collect Message Tracking Logs. Results for Get-MessageTrace are gathered in records per page. This "records per page"
    # is configureable via the -PageSize parameter using a value ranging from 1 to 5,000 with a default of 1,000. Without
    # your result count you cannot specify the -Page parameter's value, so we need to collect all pages via a loop.
    $Messages = $null
    $Page = 1
    #Write-Host "Collecting Message Tracking for"$StartDate
    do
    {
    #Write-Host " …Page $Page"
    $CurrMessages = Get-MessageTrace -Page $Page -StartDate "$StartDate 00:00:00" -EndDate "$EndDate 00:00:00" `
    | Select-Object MessageId, Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageTraceId, Index

    $Page++
    $Messages += $CurrMessages
    }
    until ($CurrMessages -eq $null)

    # Export Resulting Messages to CSV (explore options for using switch instead of elseif)
    $Year = ((Get-Date $StartDate).Year)
    $Month = ((Get-Date $StartDate).Month)
    $Day = ((Get-Date $StartDate).Day)
    $Zero = 0
    $CsvOutput = "$env:USERPROFILE"+"Email Activity Reports"
    Do {
    # "0" pad single digit month and day
    If ($Month -lt 10 -and $Day -lt 10) {
    $Messages | Export-CSV "$CsvOutput$Year$Zero$Month$Zero$Day.csv"
    $CsvOut = 'True'
    }
    # "0" pad single digit month only
    ElseIf ($Month -lt 10 -and $Day -gt 9) {
    $Messages | Export-CSV "$CsvOutput$Year$Zero$Month$Day.csv"
    $CsvOut = 'True'
    }
    # "0" pad single digit day only
    ElseIf ($Month -gt 9 -and $Day -lt 10) {
    $Messages | Export-CSV "$CsvOutput$Year$Month$Zero$Day.csv"
    $CsvOut = 'True'
    }
    # double digit month and day
    ElseIf ($Month -gt 9 -and $Day -gt 9) {
    $Messages | Export-CSV "$CsvOutput$Year$Month$Day.csv"
    $CsvOut = 'True'
    }
    }
    While ($CsvOut = $null)

  9. # Optionally output messages to GridView output sorted by Received column from StartDate to EndDate
    # just replace those $Messages | Export-CSV "… lines with the following
    #$Messages | Sort-Object Received | Out-GridView -Title "$StartDate Message Tracking Data"

    I found your site, and this post looking for hints and ideas for an upgraded version of this – whenever I do a big project like this I like to look for any tips and tricks others may be sharing – I think it is pretty lame to re-invent the wheel and call yourself
    an inventor LOL so in the heart of full disclosure this code is the result of many internetizen's writings, but not any one in particular, best help for me typically comes from Glen's Exchange Dev Blog over at
    http://gsexdev.blogspot.com. This was a fun challenge, I need another 😉

    Just thought I would share…I find myself oddly enough, with a moment to do so.

  10. OK, I thought about it. Admins need a way to automatically pass credentials via PowerShell to run unattended scripts on a schedule. Here is what I do to accomplish the credentials part – that free time I had is quickly evaporating so you are on your own
    for scheduling 😉
    Beyond securing the location where I store the encrypted string file that is my Office 365 password I also use just a little obfuscation of the file’s name to prevent any other admins from seeing “password” or “Office 365 Super God Mode Password” and just tempting
    them beyond reason.

    $SecureData = cat C:directoryo365.txt | convertto-securestring

    $365Cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "email@contoso.com",$CatalogData

    Import-Module MSOnline

    $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri
    https://ps.outlook.com/powershell -Authentication Basic -Credential $365Cred -AllowRedirection

    Start-Sleep -Seconds 5
    Import-PSSession $Session

    Start-Sleep -Seconds 10
    Connect-MsolService -Credential $365Cred

    I do those sleeps only because I find when I automate this, there are times where the session and msolservice are not properly loaded.

    How to make the file used with the $SecureData variable? Here you go!
    Read-Host “Type your Office 365 Exchange Online password: ” –AsSecureString | ConvertFrom-SecureString | Set-Content C:directoryo365.txt
    Here’s the catch – when you run this command you want be logged in to Windows with the user account that will run the automated script. I have not tested this in depth, i.e. user1 on computer2 creates file and user1 on computer1 executes script – this may work
    if we are talking about a domain user, local user? I doubt it, but again…I have not tested. I just do this on the computer that will run the script and logged in with the user the script runs as.

    I like to really understand things and sometimes, my mind just does not speak MSDN so here's a good writeup I just found for you all…
    http://knowledge.quickstart.com/powershell-credentials-and-securestrings
    http://knowledge.quickstart.com/powershell-credentials-and-securestrings-part-ii
    http://knowledge.quickstart.com/powershell-credentials-and-securestrings-part-iii

  11. UCWARRIOR says:

    Hi,

    Is it possible to trace who has been sending IRM encrypted messages?

  12. Jeff25 says:

    You intro talked about "7 day" trace, but you do not show this. You only show a 48 hr trace.

  13. Joe Frixon says:

    If we need to trace email from past 90 days are there any modification which can be done here?

  14. Jay Thakker says:

    When I run the below script, it runs and shows the results properly but the exported CSV file is blank.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddDays(-30)

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageID, MessageTraceID | Out-GridView | Export-CSV D:resulttest.csv

    What is it that I am not doing right in order to export the results of past 30 days into CSV?

  15. Hi Jay, if you want export the data to a CSV, remove the Out-GridView from your script. It should instead look like this.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddDays(-30)

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageID, MessageTraceID | Export-CSV D:resulttest.csv

  16. Eric Martinez says:

    Hi Andrew…I need to figure out which distribution groups have NOT received any emails in the last 7, 14, 21, 30 days. Is this something that can be done with PowerShell? I'm a newbie to PowerShell and if can point me in the right direction it would be
    greatly appreciated. The environment i'm working with has over 2000 distros and I've been tasked with cleaning it up. So any distro that has not received emails in over 30 days will be removed from Exchange.

  17. Hi Eric, that’s a great question. Unfortunately nothing straight forward comes to the top of my mind. If I figure out something I’ll post back here.

  18. Joshua McCorkle says:

    Hi Andrew, thanks for the information.

    I noticed when I use either the AddDays or AddHours I only get the messages from a single day, not the range of dates.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddDays(-5)

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject, Status, ToIP, FromIP, Size, MessageID, MessageTraceID | Where {$_.Status -eq "Failed"} | Out-GridView

  19. Dave says:

    Has the ability to search by Subject changed recently?

    $dateEnd = Get-Date
    $dateStart = $dateEnd.AddHours(-48)

    Trying to find emails that has the phrase "Fax Received" in the last 48 hours.

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject

    This by itself, gives me a list of all emails in the last 48 hours (I can visually see several emails with the phrase I’m looking for)

    Adjusting my query to

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject | Where {$_.Subject -eq "Fax Received"}

    And I’m getting 0 results back. Tried different variations such as "*Fax Received*" as well as different operators than -eq, like -contains but nothing is giving me results.

    Just wondering if I’m off in my syntax or if this particular type of subject search doesn’t function the way I thought it once did.

    Thanks! Fantastic blog by the way!

  20. Hi Joshua, I just ran your PS on my tenant and I’m seeing results for 5 days in the past. If you run a regular message trace through the Portal and select Failed messages in the last 7 days, do you get more results?

  21. Hi Dave, thank you for the feedback! Your PowerShell looks ok. The searching donw with Where-Object is pure PowerShell and doesn’t have dependencies on the Exchange Online PowerShell. Try the following and let me know if it returns anything.

    #Set these accordingly:
    $dateEnd = Get-Date
    $dateStart = $dateEnd.AddHours(-48)

    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Select-Object Received, SenderAddress, RecipientAddress, Subject | Where {$_.Subject -like "*Fax*"}

    In my test tenant, my subject searching using this syntax is currently working. Also try to filter on different subjects that are present to see if the problem appears for all subject searching.

  22. Dave says:

    Perfect. I believe I was mixed up in my operators that I was trying to use. Combine this with the PageSize parameter and I'm getting the results I need. Thank you!

  23. Josh says:

    Hello! I was wondering if there is any way to run the trace for certain users pulled from a CSV, and return the results all into the same output?

  24. Nathan S. says:

    Are there known limits of the number of messages message trace can retrieve in one go? I'm trying to see if one of our addresses is bumping up on the EOL recipient limits for a 24 hours period, but only seem to be getting 1,000 results, when I'm pretty
    sure there are at least 2x that should be shown. Or maybe it's a timeout limit?

    Thanks!

  25. Hi Josh, you should be able to do something like that with PowerShell. I don’t have all of the specifics from your ask, but it should be completely possible to script that.

  26. Hi Nathan. By default, only 1000 results will be returned. You can set the result size from anywhere between 1 to 5000. Just use the -PageSize parameter and set a value larger than 1000. See
    https://technet.microsoft.com/en-us/library/jj200704(v=exchg.150).aspx.

    If there are more results than what your PageSize is set to, you can specify -Page in your call which will indicate which page number you want to see. There’s more explanation of this at the above link. Let me know if you have any problems with this.

  27. Max says:

    How can I get more than 1000 records returned in a message trace, no -resultsize unlimited option exists?

  28. TechNiels says:

    Very helpful, thank you.

  29. Robert Strom says:

    Andrew,

    I was on the phone with Microsoft support earlier this week and they specifically told me that searching by subject was not possible at this time yet another support person pointed me to this page.

    I’ve tried what are essentially your exact commands as shown below for querying in PowerShell by subject and they are not working for me. I know that the messages exist because I can search for them using -SenderAddress and see the results yet when I search for one of those subjects I get no results. Here are the exact commands that I am running.

    $dateEnd = get-date
    $dateStart = $dateEnd.AddHours(-48)
    Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Where {$_.Subject -eq “Fw: IRS inquiry for mycompany.com”} | Out-GridView

    I am running Windows 10. PowerShell reports version 5.0.10586.494

    Any assistance you can provide here would be greatly appreciated.

    Thanks,

    Robert

  30. clarion says:

    I used the days function set 45 days. received the message cant be more than 30. I really want to search a sender and subject but cant seem to find any info on that.

  31. Tom Oriel says:

    This has been a huge help… thanks.
    I’m trying to run this in a foreach loop where I feed in specific contact email addresses. I need to search for blocked emails sent from a specific group of contacts.

    $MainContacts = Get-MailContact -ResultSize unlimited | Select-Object primarysmtpaddress
    $dateEnd = get-date
    $dateStart = $dateEnd.AddDays(-1)
    Foreach ($Contact in $MainContacts)
    { Get-MessageTrace -StartDate $dateStart -EndDate $dateEnd | Where {($_.SenderAddress -eq “$Contact”) -and ($_.Status -eq “Failed”)} }

    Any idea what I’m missing?
    Thanks

  32. Tim Welford says:

    This is great. Once I have my list is is possible to get the message content from a failed message via Powershell?

  33. Andy T says:

    Very useful thanks! 🙂

  34. Great! but i have a problem, i got the result in UTC time zone and i have it configured in UTC-05:00 how can i fix this

Skip to main content