Public Folder Replication Fails Due To Empty Legacy Administrative Group

EDIT 10/18/2010: Removed the reference to upcoming fix (no fix current planned).

Some customers are finding that when they try to replicate their public folder content to Exchange 2010, it will not replicate. Looking at the application log, you may find an event like this:

Log Name: Application
Source: MSExchange Store Driver
Event ID: 1020
Level: Error

The store driver couldnt deliver the public folder replication message “Hierarchy (” because the following error occurred: The Active Directory user wasn’t found.

This happens because we assume that if a Servers container exists, there will be a System Attendant object somewhere inside it. However, in an environment where Exchange 2000 or 2003 previously existed, and all those servers have been removed, you probably have a First Administrative Group or some other Administrative Group still hanging around with a Servers container, but no servers inside it. When the Exchange 2010 Store Driver stumbles across this empty Servers container, this error is the result.

To work around the issue, get rid of that empty Servers container. Exchange System Manager won’t let you delete it, but you can use the ADSI Edit tool to remove it. Just be sure that the Servers container you are deleting is one that has nothing in it!

This workaround invariably brings up the question: Can’t we just delete the old Administrative Group since we don’t need it anymore? After all, Exchange System Manager will let you gracefully delete an Administrative Group. The answer to that is a little more involved.

The current documentation on TechNet strongly recommends against deleting old admin groups, and it states that if you do, and you see mail backing up in the System Attendant mailbox, you must then use ADModify to change the legacyExchangeDN on all your users from the old admin group. However, this is not entirely true, and that documentation is being changed.

Items stuck in the System Attendant mailbox

The System Attendant is responsible for publishing free/busy messages to public folders for certain clients such as OWA. When it can’t publish these items to the appropriate free/busy public folder, the items stay in the System Attendant mailbox until the problem is corrected.

The most common cause of this problem is that there is simply no replica of the free/busy folder for a given admin group, which usually happens when the public folder stores for that admin group have been ungracefully removed (such as deleted through ADSI Edit) without properly replicating all the content off. This actually has nothing to do with the removal of the admin group itself, but rather with the removal of the public stores in that admin group.

Fortunately, this problem is easy to correct. Forcibly deleting all the public stores in an admin group does not delete the free/busy folder from the hierarchy – it just means all the replicas are gone. To correct this, you can use your public folder admin tool of choice to navigate to the System Folders and find the free/busy folder in question. Then, just add a replica of that folder to a public store in some other admin group. After allowing time for replication, the System Attendant mailbox should clear.

The important point here is that mail backing up in the System Attendant mailbox is not a reason to change the legacyExchangeDN on the users from that old admin group. Just add a replica, and you should be good to go.

A missing free/busy folder

Removing a legacy admin group does not typically cause a problem. However, there is a possible situation to be aware of.

Every admin group has a siteFolderServer attribute. This attribute points to the public folder store that is responsible for the free/busy folder for that admin group. Most of the time, that public folder store doesn’t have to do anything, but it is responsible for making sure that the free/busy folder exists. If the free/busy folder for that admin group is missing, it’s up to that public folder store to create it.

You can’t delete the free/busy folder through any admin tool (ESM, the cmdlets, etc, won’t let you), or even through something like ADSI Edit – it’s an object in the store, not in the Active Directory. However, it is theoretically possible that somehow, that folder could go missing. If you deleted all your public folder databases and started over with clean ones, or something else unusual happened, you could end up in a situation where there is no free/busy folder for a particular legacy admin group. If that legacy admin group no longer exists in the directory, and thus has no siteFolderServer specified, then the free/busy folder will not get recreated, and you’ll see items backing up in the System Attendant.

Even in this situation, there’s a fairly easy way to fix the problem. If you still have Exchange System Manager, you can use it to recreate the legacy admin group. Alternately, you can use ADSI Edit to do the same. The important thing is to make sure the legacyExchangeDN is correct – make sure it matches the legacyExchangeDN on the users that were created in that old admin group. On the new admin group object, make sure you have a siteFolderServer that points to an existing public store in some other admin group. Within 24 hours, the free/busy folder for that admin group will get recreated.

If you don’t want to recreate the admin group, then your other option at this point is to change the legacyExchangeDN on the users from that admin group. The steps for this are still documented in TechNet.

Our recommendation

We recommend you leave the old admin groups around, simply because there’s no reason to remove them. Also, it’s possible your free/busy folder could go missing at some point, and then you either have to recreate the admin group or change the legacyExchangeDN on the users.

If you decide to remove an admin group anyway, you should always do it through Exchange System Manager, which will prevent you from deleting it if it still contains objects you need – like the public folder hierarchy object. Deleting the admin group while it still contains the hierarchy object will completely break public folder replication and your ability to administer the folders, among other things.

For these reasons, our recommended workaround for the public folder replication issue is to delete the empty Servers container using ADSI Edit. But technically, yes, you could delete the admin group – gracefully from ESM – to achieve the same end. This doesn’t usually cause a problem, and situations where you have to change the legacyExchangeDN of the users should be pretty rare.

Bill Long

Comments (15)
  1. JohnB says:

    So what happens to users’ LDN if the old admin group is removed and those users have migrated from E2k3 to E2k7 to E2010? Doesn’t the LDN still point to the old admin group? Won’t any application (third party appls like RIM for that matter) that utilizes the LDN for amilbox lookup break?

  2. B.M says:

    Funny. After deleting the CN=Servers i get still an NDR back when I try to send a email to an email enabled Folder in Public Folder. I get something like this:

    #550 5.2.0 STOREDRV.Deliver: The Microsoft Exchange Information Store service reported an error. The following information should help identify the cause of this error: "MapiExceptionNotFound:16.18969:B1000000, 17.27161:0000000054000000000000000000000000000000, 255.23226:00000000, 255.27962:44000000, 255.17082:BFF9FFFF, 0.25697:00000000, 4.21921:BFF9FFFF, 255.27962:FA000000, 255.1494:35000000, 255.26426:96000000, 4.18236:BFF9FFFF, 4.7974:BFF9FFFF, 255.1750:43000000, 0.26849:43000000, 255.21817:BFF9FFFF". ##

  3. JH says:

    I have the same problem as B.M. I opened a call at PSS – but it’s over a week now and they still have no clue.

  4. Justin King says:

    Sometimes I wonder if it’s a subtle nudge to get us off Public Folders ;).

     I’m 4/4 Of 2010 migrations where the empty servers container causing routing issue has emerged, so I’m VERY glad that there is now a somewhat official link i can reference future clients to assure them I’m not randomly hacking around with ADSIedit and that this is a MS acknowledged problem and fix (though a KB article would be wonderful, even if you dont provide a hotfix)

    It does introduce its own problem, however:

    The BPA for 2010 will report a configuration error if it can’t find the servers container for the legacy group, despite this being the recommended fix.  Combine that with the already existing false positive of the schema extension being an unknown revision and the BPA tool is quickly becoming unreliable.  Along with regular rollups, Exchange 2010 is in desperate need of some new BPA definitions.

    Finally, one parting shot:  please stop pressing the de-emphasis of public folders until you have a suitable replacement built-into exchange?    It’s tough to tell people they now have to purchase and deploy a whole new platform just to use a functionality they currently enjoy for free.  I get the replication model blows.  I get that it’s dated and obsolete.  Let it die.  But not before you have an Exchange native answer.  It hurts those of us consultants who are trying to sell this otherwise fantastic platform.

    OK parting shot taken … keeping pressing forward on what is clearly the best Exchagne release to date …. and Ill promise to ride you less on the rough edges ;)

  5. bilong says:

    JohnB, yes the legacyExchangeDN will still point to the non-existent admin group. But no, nothing should break. Ever since Exchange 2000, you’ve been able to move mailboxes to a new admin group and delete the old one. That was a feature of Native Mode move mailbox, and it doesn’t change the legacyExchangeDN. Because of that, it’s actually quite common and normal to have legacyExchangeDNs on mailboxes that point to an admin group which no longer exists.

    B.M, that’s a different problem with a few causes. One cause is simply not having permissions. You must have at least Contributor rights to the public folder. Another cause is an improperly formatted P2 header (the "To:" line) when the email is coming from an external sender. For instance, if the To: line contains just a display name and no actual email address, this happens. Yet another cause is hierarchy replication failing so that we don’t see the folder on certain stores. If you can’t track down the problem, get a case open so we can find out what’s going on.

    JH, I’m trying to find your open case so I can review it, but I’m not having much luck. If you click on my name to go to my TechNet blog and then click on the Email link there to contact me, I’ll look into it.

  6. Dave Stork says:

    It’s not only the PF replication that has this issue, also mail enabled public folders generate a similar message.

    I had it after a Exchange 2003 transition to 2010. See my blog posting about it:

  7. Darko Bazulj says:

    Exchange 2003 to 2007 or 2010 generates the BFF9FFFF when it cannot understand the translation between an Bridgehead and CAS. If you manually telnet to the SMTP of the old 2003 Front-End try emailing the mail-enabled PF, it will work. This is due to the CAS being excluded. You should be able to get around this issue by replicating the 2003 folder to the 07/10 server. Once if fully replicates the CAS server will no longer create the error. Hopefully this helps with some of the discussion.

    For the other part of the discussion on the TO line, you need to create a hub transport rule to remove the To header from the emails. It may be a pain, but I believe I read somewhere that this is a known bug.

  8. Robert says:

    This is one of those "how did this not get caught in testing bugs" IMHO.  Many of of wasted hours and hours of time tring to fix this assuming it was "our" fault…

  9. Sean says:

    My issue is I did not "Move" the Public Folder hirearchy from Exchange 2003 to the 2010 admin group/server prior to removing the last 2003 server.  How can I do this after the fact?  Also could you link to how to re-create legacy Admin Group via ADSI Edit?


  10. Darko Bazulj says:

    Ok, so this is a critical step in removing the last 03 server. I can’t imagine there is an alternate step other than to recreate the PF in 2010, possible export the items out of the existing PF, reimport to new PF and then re-replicate.  As far as recreating the legacy Admin Group, if you have a backup you could possibly complete an Authoritative Restore, but normally this also causes other corky problems. Here is a good thread that talks to a similar case in technet land. Hopefully some of these steps can help you.

  11. WorkingHardInIt says:

    If I have multiple legacy Severs containers do I need to remove them all? I deleted the First Administrative Group one but that did not seem te ofix the issue. In the lab with only one legacy servers container it did.

  12. Tricks says:

    I have a  frustrating issue with replication failures of PF Hierarchy and Content from a single exch 2003 to a single exch 2010 (CAS,HT,MB roles), I have trawled through a lot of postings with similar problems but NO answers to help my situation (this is the simplest topology one can wish for). I have increased the logging level and can see the source server sending the repl messages but they get stuck with the categoriser. MSExchnage transport event id 6015 followed by 9013, are logged on the exch 2003 application log. I have seen some posts that answer these errors but they relate to pre -exchange 2010.  Can any one point me in the right direction please.

  13. Chris says:

    I’m having the same issue at B.M and JH after a migration from 2003 to 2010 having deleted the emtpy servers container, sending to a migrated mail enabled public folder results in the following NDR:

    550 5.2.0 STOREDRV.Deliver: The Microsoft Exchange Information Store service reported an error. The following information should help identify the cause of this error: "MapiExceptionNotFound:16.18969:B4000000, 17.27161:0000000054000000000000000000000000000000, 255.23226:00000000, 255.27962:44000000, 255.17082:BFF9FFFF, 0.25697:00000000, 4.21921:BFF9FFFF, 255.27962:FA000000, 255.1494:00000000, 255.26426:96000000, 4.18236:BFF9FFFF, 4.7974:BFF9FFFF, 255.1750:C0010000, 0.26849:35000000, 255.21817:BFF9FFFF". ##

    Is there a fix yet?

  14. Chris says:

    Decided to open a case with PSS on this issue. No fix as such but the issue was resolved by exporting the PF to a PST file, deleting the PF and recreating/importing the PST.

    The PSS rep said he’d had the issue before as well so seems likely to be a bug.

  15. Gary Enticott says:

    This is poor, Very poor.

    why was this not tested before releasing this to RTM.

Comments are closed.