Messaging Records Management & confusion with Litigation Hold

  • Article

Many customers I speak with are confused by the terminology used with Messaging Records Management (MRM).  Often people assume that if you set a Retention Policy the item will be kept for the number of days specified.  For them this implies the items in question will also be protected for that period of time.  This blog's aim is to review the terminology used by MRM and distinguish what MRM can and cannot do.

To help illustrate the differences I will use an example.

A few months ago I had a customer who thought items were being kept in the mailbox longer than intended.  He had set up a Default MRM Deletion tag to remove items that were more than 3 years old.  He had also turned on Litigation Hold on the mailbox.

Here are the properties from the mailbox:

LitigationHoldEnabled                  : True

SingleItemRecoveryEnabled              : True

RetentionHoldEnabled                   : False

LitigationHoldDate                     : 6/7/2016 9:54:28 AM

LitigationHoldOwner                    : admin@contoso.com

LitigationHoldDuration : Unlimited

RetentionPolicy                        : Default MRM Policy

 

Here are the properties of the 3 year delete retention tag:

WhenChanged                           : 2016-12-05 9:42:54 AM

WhenCreated                           : 2016-06-29 1:39:10 PM

WhenChangedUTC                        : 2016-12-05 2:42:54 PM

WhenCreatedUTC                        : 2016-06-29 5:39:10 PM

 

 

In this case the Litigation Hold was applied before the retention policy. One part of the misunderstanding was that the customer thought the Retention setting from the Default tag took precedence over the Litigation Hold. Unfortunately it is exactly the opposite. In this case the words got in the way. The world outside Microsoft gives a different meaning to retention than the developers of Exchange 2010 and MRM gave to it back when this was designed during development back in 2008.
A few definitions/explanations:

 

Litigation Hold is a blanket that is thrown over the entire mailbox.  When the database engine is asked to permanently remove an item from the Exchange database there is a check to see if the mailbox is protected by Litigation Hold.  If the answer is Yes the item cannot be deleted.  If the answer is No the deletion proceeds and the item is gone forever.  You might say it is a mail item's last chance to avoid destruction.  Litigation Hold ensures an item cannot be permanently removed from the mailbox when it is Younger than the specified age (specified in the LitigationHoldDuration property).  For everything except Calendar Items and Tasks that age is calculated by the date/time at which the item was received.  If there is no Receive Date then the creation date is utilized for the calculation.  Calendar Items and Tasks use the last occurrence as the basis for the calculation instead of the received date.

 

In-Place Hold has the same effect on permanent deletion of items that Litigation Hold does.  However, the scope of an In-Place is different.  Where Litigation Hold applies to EVERYTHING in the mailbox an In-Place hold utilizes a query to protect items.  For example, Lets assume that only Invoice related items are considered important enough to keep for 3 years.  We could create an In-Place hold that checks the Subject and Message body for the word "Invoice".  IF the word "Invoice" is present on the item its permanent removal from the mailbox will be prevented for 3 years.  If the word is absent the item can be removed as if the In-Place Hold was not there.

 

Retention Policies (aka Messaging Records Management (MRM)) are named a little counter intuitively.  They actually do the opposite of retaining items.   MRM specifies how old the item can be before it is deleted or archived.  One way to look at it is to compare it to the expiry date stamped on some grocery items. The expiry date on the items suggest when they should be thrown away. There is nothing to stop you from throwing them away sooner. The date does not guarantee the item will stay in your kitchen until the date specified. You can almost look at MRM as an automated process that cleans out the kitchen's expired items. MRM does nothing to prevent disposal of the item earlier than the expiry date, that is the job of the legal holds described above.

The original name for MRM before Exchange 2007 released was Email Life Cycle (ELC).  You will still see ELC in some of the results from PowerShell cmdlets and Microsoft's internal documentation.  MRM consists of 3 types of tags.  Each has a different scope and order of precedence.  Here is a summary of each:

Personal Tags - These are manually applied to an item by the user by right-clicking the item and assigning the personal tag (often called a policy in Outlook).  Personal tags can be applied to most items inside a mailbox.  Because they are manually applied by a user they take precedence over the other two types of tags.  It is assumed the user knows best when they apply this type of tag.  Personal tags can archive items to the Archive Mailbox or they can order the deletion of an item.

Folder Tags (aka RPT tags) - Folder tags can only be applied to the default folders that appear in all Exchange Mailboxes.  The only action they can carry out is delete.  They are considered to be more important than a default tag.  If a folder tag is created it applies to the folder in Both the primary mailbox and the Archive Mailbox.

Default Tags (aka DPT tags) - These are applied when an item is not subject to either of the previous tags.

 

What does RetentionHoldEnabled do?

If you want to suspend MRM's functionality (prevent any of its tags from being acted upon) you can run:

Set-Mailbox myuser@contoso.com -RetentionHoldEnabled $True -EndDateForRetentionHold 12/12/2017 -StartDateForReten
tionHold 11/11/2017

The start and end dates are optional if you want an indefinite hold (just remember to remove it before calling support about a MRM policy that is not working).

To use the analogy above this will keep the items in the fridge until after the EndDateForRetentionHold.  This is useful if you know someone is going on leave for an extended period.  It allows you to suspend MRM until they have been back for a little while and thus give the user a chance to catch up.

 

Recoverable Items\Deletions - This folder maps to the Recover Deleted Items functionality in Outlook and OWA.  Any item in this folder can be recovered with the standard mail clients.  How long items stay in this folder is governed by the RetainDeletedItemsForproperty of the mailbox (this property is visible when you output the results of get-mailbox).

Recoverable Items\Purges - This folder houses items that have been deleted for more than RetainDeletedItemsFor days AND that are protected by Litigation Hold.

Recoverable Items\DiscoveryHolds - This folder houses items that have been deleted for more than RetainDeletedItemsFor days AND that are protected by In-Place Hold.

 

 

The Life cycle of an email

 

For all of these examples we will make three assumptions:

Single Item recovery is enabled for the mailbox (the default in Exchange Online)

The RetainDeletedItemsFor property of the mailbox is set to 14 days with this value:  14.00:00:00

When a Litigation Hold or In-Place Hold is used the duration of this hold is 3 years.  The 3 years is recorded as 1096 days to account for the possibility of a leap year.

 

 

Scenario 1:   A Mailbox with no Retention Policy or Litigation/In-Place Hold.  User Just reads the Item.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time and never takes action on the item again.
  • Today this item would still be sitting in the Inbox.

 

 

Scenario 2:   A Mailbox with no Retention Policy or Litigation/In-Place Hold.  User deletes the item a few weeks after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • The user Shift deletes the item on April 3rd 2012 at 20:05:52.574.  The mail item moves from Inbox to Recoverable Items\Deletions at the time of the Shift+delete.  Shift+Delete is considered a Hard Delete.  Without SingleItemRetention=$true the item would disappear from the mailbox instantly.  Since SingleItemRetention is the default behaviour for Exchange Online mailboxes the message moves to the Deletions folder.
  • On April 17th, 2012 at 20:05:52.575 the item is no longer protected by the RetainDeletedItemsFor property.  The database engine permanently removes the item within a few minutes.

 

 

Scenario 3a:   A Mailbox with no Retention Policy.  The mailbox has a 3 year Litigation Hold.  User deletes the item a few weeks after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • The user Shift deletes the item on April 3rd 2012 at 20:05:52.574.  The mail item moves from Inbox to Recoverable Items\Deletions at the time of the Shift+delete.  Shift+Delete is considered a Hard Delete.
  • On April 17th, 2012 at 20:05:52.575 the item is no longer protected by the RetainDeletedItemsFor property.  The database engine attempts to permanently remove the item within a few minutes.  The removal attempt cannot proceed because of the Litigation Hold.  The server does not want to leave the item where it can be recovered by Outlook or OWA.  Therefore it moves the item to the Purges folder.
  • On March 2nd, 2015 at 15:37:16.715 UTC time the protection of the Litigation Hold ends.  The database engine permanently removes the mail item from the Purges Folder.

 

 

Scenario 3b:   A Mailbox with no Retention Policy.  The mailbox has a 3 year Litigation Hold.  User deletes the item a little more than 5 years after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • The user Shift deletes the item on April 3rd 2017 at 20:05:52.574.  The mail item moves from Inbox to Recoverable Items\Deletions at the time of the Shift+delete.  Shift+Delete is considered a Hard Delete.
  • On April 17th, 2017 at 20:05:52.575 the item is no longer protected by the RetainDeletedItemsFor property.  The database engine attempts to permanently remove the item within a few minutes.  The removal attempt succeeds because the Litigation Hold only protected this item for 1096 days from its arrival on March 1st 2012 at 15:37:16.714 UTC time.  Therefore any delete of the item at or after March 2nd, 2015 at 15:37:16.715 UTC time proceeds as if there was no Litigation Hold in place.

 

 

Scenario 4a:   A Mailbox with a 3 year delete Retention Policy.  The mailbox has a 3 year Litigation Hold.  User deletes the item a few weeks after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • MRM stamps the item with a retention date of March 1st 2015 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • The user Shift deletes the item on April 3rd 2012 at 20:05:52.574.  The mail item moves from Inbox to Recoverable Items\Deletions at the time of the Shift+delete.  Shift+Delete is considered a Hard Delete.
  • On April 17th, 2012 at 20:05:52.575 the item is no longer protected by the RetainDeletedItemsFor property.  The database engine attempts to permanently remove the item within a few minutes.  The removal attempt cannot proceed because of the Litigation Hold.  The server does not want to leave the item where it can be recovered by Outlook or OWA.  Therefore it moves the item to the Purges folder.
  • On March 2nd, 2015 at 15:37:16.715 UTC time the protection of the Litigation Hold ends.  The database engine permanently removes the mail item from the Purges Folder.  In this instance the 3 year delete policy has no effect on how the item is handled.

 

 

Scenario 4b:   A Mailbox with a 2 year Default Permanent Delete tag in the MRM policy.  The mailbox has a 3 year Litigation Hold.  User forgets the item after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • MRM stamps the item with a retention date of March 1st 2014 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • MRM's ManagedFolderAssistant completes at 15:30 on March 1st, 2014.  The item is not deleted this day because MRM completed a few minutes before the expiry time of 15:37:16.714 UTC.
  • MRM get throttled on each of the next 4 days and terminates for the day without deleting the item.
  • The deletion that moves the item from Inbox to Recoverable Items\Purges takes place on March 6th 2015 at 15:29:28.520 UTC time.
  • On March 1st 2015 at 15:37:16.714 UTC time the item is no longer protected by the 3 year litigation hold.  The database engine deletes it within minutes.

 

Scenario 4c: A Mailbox with a 3 year Default Permanent Delete tag in the MRM policy. The mailbox has a 3 year Litigation Hold. User forgets the item after reading it.

  • An item arrives in the mailbox on March 1st 2012 at 15:37:16.714 UTC time.
  • MRM stamps the item with a retention date of March 1st 2015 at 15:37:16.714 UTC time.
  • The user reads the item on March 10th 2012 at 02:19:47:917 UTC time.
  • MRM's ManagedFolderAssistant completes at 15:30 on March 1st, 2015. The item is not deleted this day because MRM completed a few minutes before the expiry time of 15:37:16.714 UTC.
  • MRM get throttled on each of the next 4 days and terminates for the day without deleting the item.
  • The Managed Folder Assistant (MRM)  moves the item from Inbox to Recoverable Items\Purges on March 6th 2015 at 15:29:28.520 UTC time.
  • On March 20th 2015 at 15:29:28.521 UTC time the item is no longer protected by the RetainDeletedItemsFor (this mailbox property defaults to 14 days). The database engine deletes it within minutes. The 3 year litigation hold does not prevent the removal from the Purges folder because the Litigation hold only protected the item from deletion between March 1st 2012 at 15:37:16.714 UTC and March 1st, 2015 at 15:37:16.714 UTC

 

None of these scenarios cover the item moving through the deleted Items folder.  This is partly because it changes nothing with regard to the operations I am trying to demonstrate and partly because this post is just getting to be too long. 😊

 

Going back to the initial example near the top of this post...
Assume the customer's intention had been to purge all items more than 3 years old in the mailbox, and then begin a litigation hold of unlimited duration. They should apply the MRM policy to the mailbox first. For the retention policy to be effective I would recommend that the tag and policy be created and applied to the mailbox at least two weeks before the Litigation Hold is applied.  Messaging Records Management (MRM) tries to run once per day, but Microsoft only supports one completion per week.    It is a heavily throttled process. MRM is stopped any time the server shows that it is too busy. Terminating MRM removes an extra task may hinder the experience of users connected to the server.  I recommend two weeks instead of one as it often takes two complete executions of MRM to purge items.

 

 

Chris Pollitt