How do Exchange Server rules work?


You've been using Outlook for years now and you've probably established some delivery rules for yourself. Maybe mail from your manager is automatically routed to a subfolder of your inbox. If you're using Outlook 2003 or later, you've got an automatically maintained rule to handle junk mail. If you've ever set your Out Of Office, that's handled by a rule too.

But how do rules actually work? What's the implementation like? Obviously, I won't get into the grimy details (trade secrets and all), but I can pass along the basics.

First, the rules infrastructure is fully documented on MSDN. Those among you brave enough to explore this documentation may be inspired to write a little app which will enumerate your inbox rules and print them out on the console for detailed inspection of just what Outlook creates. I wrote such an app for internal diagnostic purposes (which means I most likely can't share it with you, sorry). Oh wait! Someone else has already published something here!

A rule consists of two fundamental properties, and a bunch of other support properties which mostly won't matter for this discussion. The important ones are the rule's conditions, and the rule's actions. Simplistically, if the incoming mail passes the restriction, the rule engine subsequently carries out the actions. The rule engine itself is completely agnostic to the intentions of the rule creator. In other words, the rule engine doesn't care what the title of the rule type in Outlook's wizard is – it only applies the restriction and then if that passes, it carries out the actions.

The conditions are expressed as a MAPI Restriction. As each mail is delivered, the restriction on each rule is applied to the incoming message. The restriction describes a complex property value comparison tree, such as (the received time > somedatetime AND the sender's display name starts with "Bob" AND the body contains "Loser") OR (the message size is greater than 2k). Such restrictions are also used for searching for messages, filtering views, and so on. In this case, the restriction is applied to the incoming message. This means that rules cannot check against dynamic values, such as the current system time or how big the mailbox is. It's constrained to checking properties of the message against constant values. One exception to this is how the Junk Mail rule works, which I'll describe later.

Outlook has a variety of pre-canned restrictions that it uses and I won't go into the exact nature of each one (yes, I know, you're just burning to know what the precise nature of each of these is, but you can use the sample app I linked to above to study the precise nature of each of the rule types). I will point out an important design point though: rules which are of the form "FROM some DISTRIBUTION GROUP" actually means "if the sender is a direct or indirect member of that DL". This often confuses people because they create a rule like "mail from <my team DL>, move to the <my team> folder". What ends up happening is all mail from anyone on the "my team DL" starts having their mail moved to that folder. If you think about it, it makes complete sense. However, more often than not, folks really wanted the rule "mail SENT TO <my team DL>, move it".

If the message meets the restriction criteria, the actions are then carried out. While Outlook rarely creates rules with multiple actions within, it's possible to have a list of actions on one rule. The possible actions are:

  • Move the message to another folder*
  • Copy the message to another folder*
  • Reply to the message (with a given template message)
  • Send an "Out Of Office" reply (with a given template message)
  • Fail delivery with a specific error code
  • Forward the message to another address
  • Delegate the message to another address (a lot like forward, but it preserves original sender info)
  • Tag the message with a specific property value
  • Permanently delete the message (Outlook doesn't use this when you create a rule with a delete action. Instead, it creates a MOVE action with the Deleted Items as the destination folder)
  • Mark the message as read
  • And finally, the big one: defer to the client to carry out client-defined actions.

The asterisk on Move and Copy refers to what to do if the server can't reach the destination database. In this case, the server turns it into a "defer to client" action where it specifies to the client a) the move or copy action and b) the destination that should be used. As you can see, the server itself is pretty constrained on the kinds of things it can do, yet Outlook offers all these wonderful actions that can be carried out. This is accomplished via the "deferred" action. The client stores in the action any relevant data it needs and the server will later notify the client that it needs to carry out a deferred action on a message, and includes whatever the client stored in the rule (such as, perhaps, the WAV data for the sound to play, for example).

Because a single rule could have conflicting actions, or even because multiple rules that may apply to the message could have conflicting actions, the server pulls off trickery to make sure everything happens correctly. For example, you may have two MOVE rules which apply to a message. You can't move a message to two different folders at once, yet that's the end result (the server treats subsequent MOVEs as if they were COPYs instead).

Exchange Server rules are only executed during delivery. Outlook provides a "Run Rules Now" feature, but that's implemented entirely by Outlook. The server is not involved. Also, realize that server rules are not ever executed after delivery is complete. If you witness spontaneous changes happening to your inbox, this is either Outlook carrying out deferred actions, or someone else logged into your mailbox is changing things under your feet.

What about the Junk Mail Rule? How does it adjust itself based on the administrator's configuration of the move action threshold? The administrator's configuration lives in Active Directory, so how could the rule dynamically retrieve this value for comparison against the Spam Confidence Level value stamped on all incoming messages? Well, there's a one-off cheat just for this. The rule itself is crafted to compare the Spam Confidence Level against the constant value -1. The rule execution engine pre-scans the restriction, looking for this specific comparison. If it finds it, it replaces the constant -1 with the current value in Active Directory, and then the restriction is applied to the incoming mail.

You may be thinking: "but what about two mailboxes, each with a reply rule that replies to the other mailbox? Won't that cause a mail storm?" Well, ordinarily, yes. However, our rules engine is smarter than that. As a rule is triggered on a message, the rule's ID is stamped into an ever-growing property of "I've already run this rule" history. Therefore, if a message "comes back", the rules engine can check to see if any particular rule is in this rule history and avoid running it again. So, if mbx1 sends mail to mbx2, which autoreplies back to mbx1, which autoreplies back to mbx2, the rules engine will detect the loop and not execute the rule that would otherwise cause another autoreply back to mbx1.

- Dave Whitney


Share this post :


Comments (16)
  1. Kevin says:

    This seems like a good place to ask this .. Is there a known issue in E2K7 RTM + Rollup 5 where sometimes two Out of Office replies are sent? One OOF is the one in Outlook; the other seems like an old one. We’ve seen several people experience this, but by no means all. The commonality for us is that these folks also use a popular non-Windows Mobile mobile device. However, we get the old/new OOF responses from our Microsoft TAM and I’m pretty sure he doesn’t have one of these non-WM devices.

  2. easy1ndian says:

    Thanks Dave, for clearing the mystery behind the rules processing. Enjoyable reading. I visited the script page you mentioned. Is there a possibility I can retrieve the rules from one exchange and run it again on a separate server thereby avoiding the need for creating the rules again?

    Thank you

  3. Chris Lehr says:

    Your RSS feed still shows exporting CAS settings as the most recent post.  I was thinking Sp1 support made things stagnant!

    Please fix!

  4. Exchange says:

    Chris,

    Which URL are you using?

    Both of those are updated:

    http://msexchangeteam.com/rss.aspx

    http://msexchangeteam.com/atom.aspx

    I know people get to them as I can see them in my Outlook RSS and I see RSS views going up on the server… do you see this on more than one machine?

    Please email me on ninob @ microsoft DOT com if there is still something weird…

  5. ML49448 says:

    Thanks Dave, the article make very good sense. It clears out alot of the questions that I’m unable to find.

  6. Dan Mitchell says:

    Don’t forget that the "How to read a rules table" link uses the old 5.5 EDK, which is a:not all that easy to find these days, b:next to impossible to link with, and c:buggy. Jason Johnston has a good description of why you want to steer clear of it here:

    http://blogs.msdn.com/jasonjoh/archive/2004/08/01/204585.aspx

    That said, the source for rule.dll is in libsrc in the 5.5 EDK, which means that (for a change) it _is_ sort-of documented.

  7. Dave Whitney says:

    easy1ndian: Not likely. For example, move rules contain the entryID of the destination folder of the move. Since no other mailbox has that entryID, the rule isn’t going to work once migrated. Reply rules use a seperate reply template message, which isn’t stored as part of the rule itself, and so migrating a reply rule will leave you with a non-functional rule once again.

    The easiest way to migrate rules from one mailbox to another is by using Outlook itself. Outlook provides a way to export your rules to a file, then later import them. Offhand, I don’t know if Outlook will let you migrate rules to a different mailbox, or if you’re required to re-import back into the same mailbox (sort of a backup/restore for your own rules).

    Note that there is no way to execute rules that don’t actually exist in the mailbox. Server-side execution of rules happens *only* during delivery, and Outlook can only "re-run" rules that it can find in your mailbox.

    Kevin: there is a known problem with that popular non-WM mobile device wrecking OOF rules. Call them up. We’ve already alerted them to their bug. There’s also a rare problem on our end (in general) which should be fixed in SP1.

  8. Richard Sobey says:

    Dave,

    Thanks – really great article. If I can, I’d like to pick your brains about the final part, and how auto-reply rules don’t create loops. Does this behaviour extend to Public Folder "Folder Assistant" rules, and/or mail sent from external organizations?

    Thanks

    Richard

  9. Dave Whitney says:

    It extends to all rule operations within an organization, and only across organizations which communicate "Exchange to Exchange". In other words, the supporting properties which the rules engine uses to detect loops may be stripped out by other mail systems. Exchange may also strip the property out on non-authenticated connections (to prevent possible malice). So, in the general case (for example, an auto-reply rule back to some internet listserv), the loop detection probably won’t work because that foreign system stands a fair chance of dropping the loop-detection properties from the mail.

  10. Kike Suay says:

    I have the same problem with OOF and non-wm device, I’m planning the update to SP1. Is realy solved in SP1?

  11. rcmore says:

    Why is assigning a message to a category a client side rule?  Can’t this be done in exchange server?

  12. Nacker says:

    Dave,

    Is there any way to write software to process incoming messages in such a way that Exchange Server rules (like OOF) won’t get applied to certain messages?  My company has an issue where some automated emails are triggering OOF emails that end up confusing and frustrating the recipients, so any ideas of how to solve this problem would be great.

  13. replacements says:

    I wanted to ask, the rule, ‘have the server reply using a specific’ message works the same way?  If so what is the KB that discribes this?

    Thanks

  14. BTW – MFCMAPI can read and modify your rules: http://codeplex.com/mfcmapi

  15. Dave Bonfield says:

    Is there *any* way at all to modify server-side rules from a PPC Mac?  This seems like a serious limitation to the use of Exchange (which sadly i have just been migrated to) in a mixed-platform environment.  

    There are no Mac browsers (as far as i know) supporting OWA premium (i tried spoofing it with Safari but it just doesn’t work), and OWA light can’t modify rules.  Entourage is reported not to be able to modify the rules either, although i haven’t tried it.  

    i was really happy with the Unix system we had before where i could just ssh in to the server, create a .forward file with a couple of email addresses in it, and i was done….  :-(

    i fully accept that this is a bit trickier for windows users who have no idea how to use a terminal, but surely there is some happy middle ground to be had, so that i don’t need a million lines of windows code porting in order to change my mail forwarding preferences?

  16. SuzyC says:

    Is there a way to see all the rules that are running on an Exchange 2007 SP1 server?

Comments are closed.

Skip to main content