HTTP 500 Internal Server Error when logging into Exchange 2013 Exchange Control Panel (ECP)

Over the past several months, I've seen an increased number of Exchange 2013 cases where certain admin users received a 500 status error when attempting to log in to ECP:

The first question I ask when I get these specific cases is "Does the admin account have a mailbox"?

 In pretty much every case that I can remember, the response is "No".  And immediately, I know exactly where to look!

 

A little history…

 As you may know, because of the design of Exchange 2013 - the CAS role simply locates your mailbox, then proxies the request back to your mailbox server - an "anchor mailbox" is used.  This anchor mailbox is simply your Exchange GUID, and is used for a number of reasons in Exchange 2013 connectivity.  The most obvious, though, are:

 

  • As mentioned above, so that an Exchange 2013 CAS knows what mailbox server to proxy a request to
  • To prevent the 'Your administrator has made a change so you must restart Outlook' message that you get when a mailbox is moved to a different site.

 

You can see evidence of the anchor mailbox when you configure an Outlook profile for an Exchange 2013 mailbox:

Notice the ExchangeGuid is used as the 'Server' in the outlook profile, instead of an actual server name.

 

OK that's great and all, but what does that have to do with this status 500 that I'm getting?

 Well, let's think about it.  If the Exchange 2013 CAS needs to know where to proxy a request, but the request is coming from an account that has no mailbox, and thus no ExchangeGuid associated with it, how does Exchange know which mailbox server to proxy?

 In instances like this, Exchange 2013 uses the ExchangeGuid of system mailboxes to determine where to proxy the requests.   In this specific scenario, Exchange uses the system mailbox SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}

Note: When pulling up the system mailbox(es) using Get-Mailbox you must specify the -Arbitration parameter or the command will not work.

 

So how do I fix this?

Typically, what I've found is that this is caused by either the database with the system mailbox is unavailable, or some attribute on the system mailbox account has become corrupt or is missing. 

Many times, this issue is accompanied by another - such as users unable to connect to Exchange, and the 500 error is discovered when the admin attempts to log in to ECP to try and figure out what is wrong.  In this case, Exchange Management Shell is your friend as you will need to determine whether the database is mounted via shell. 

In other instances, I've found that system mailbox, for some reason has incorrect information, such as missing email addresses.  You can either compare it to the other system mailboxes (Get-Mailbox -Arbitration) or compare to your lab environment (I hope you have a lab!)

Finally, I've seen some companies simply deleted the database that had the system mailboxes on it.  Or delete the AD account for the system mailbox.  Or both.  This isn't cool.  Exchange uses these mailboxes for many things, including OAB generation, so it is critical to keep these mailboxes healthy!  To fix this:

 

If the mailbox was deleted but the AD account still exists:

Run: Enable-Mailbox "SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}" -Arbitration

You may also need to specify the database parameter to assign a database.

 

If the AD account was deleted:

First, run: Setup /PrepareAD

Then run the above Enable-Mailbox command.

 

Of course, the /PrepareAD switch does a lot more than just create system mailboxes, so proceed with caution.

 

Is there a workaround?

Yup!  Since we use the system mailbox for it's ExchangeGuid, you can simply just add a mailbox to your admin account, and that should get you up and running.  I should stress again though that this is not the only thing system mailboxes are used for, so it is important to plan to fix whatever is wrong with them.

 

Thanks to Chris Thevaos for assistance with investigating the root cause of this issue.