Why Exchange Databases Might Remain Dirty After ESEUTIL /R Recovery



You’re trying to recover your Exchange database from the “dirty shutdown” state to “clean shutdown.”  You use ESEUTIL /R and after several ill-fated attempts the databases remain dirty.  What’s happening?  Danijel Klaric. a Microsoft Premier Field Engineer based in Germany, sheds some light on why the Exchange 2007/2010 databases might remain dirty after a seemingly “successful” ESEUTIL /R recovery, and provides pointers on how to solve this.  Enjoy his article!


Normally I wouldn’t think that this following topic would ever be a problem, but as I faced the issue two times in the last three weeks, I thought it would be good to shine some light on the subject.

Both customers are using Exchange 2010 with Native Data Protection, and have enabled a seven day lag on one of their database copies to enable their admins to recover items past the “single item recovery” period, or to recover deleted folder structures. Remember that single item recovery doesn’t preserve the folder structure as mails moved to the dumpster are stored in a flat hierarchy.

The Symptom

They thought they were following appropriate Microsoft guidelines to use logs needed to recover the database from the “dirty shutdown” state to “clean shutdown”, so that the database would be usable as a recovery database for extracting the needed content.

1) Collect the database and the required logs

First grab a copy of the edb file and needed logs (i.e. logs needed from database perspective and logs they would like to roll forward to). This can be accomplished by suspending the database copy, and then copying the needed files to a separate location or create a snapshot of the volume which can be reverted to at a later time.

2) Check database state

Run the following: eseutil /mh "c:\DBRecovery\Mailbox Database 0436312751.edb"

eseutil /mh "c:\DBRecovery\Mailbox Database 0436312751.edb"

Check the database header for log files generation needed for the recovery, in this case: Log Required 124-124 (0x7c-0x7c) which means file E000000007C . These log files are needed at a minimum to recover the database to a clean shutdown without data loss.

Side note: If you try to recover the database with only “Log Required” logs and NOT “Log Commited” logs it will throw the following error:

Side note: If you try to recover the database with only “Log Required logs” and NOT “Log Commited” logs it will throw the error:

You can continue with the recovery using /a. And the last committed transaction will be removed from the database and the database will have a clean and consistent state. Otherwise just add the log files mentioned in the “log committed” field to your log directory and this error will disappear.

3) Check consistency of needed logs

Before starting to apply the logs you would like to, it´s important to check if all logs are consistent and available. If you need to handle a large number of logs and have used the Windows Explorer to copy the files, occasionally I’ve seen that some files are missing when sorting them in the Windows Explorer view.

To ensure that you will not fail afterwards when performing the recovery, take a moment and check these. Use eseutil /ml together with your log directory path and log prefix, in my example E00.

Log file check within directory: eseutil /ml c:\DBRecovery\E00

Log file check within directory: eseutil /ml c:\DBRecovery\E00

4) Recover database state to “clean shutdown”

After they finished both checks successfully, they started the recovery using:

eseutil /R E00 /l "c:\DBRecovery" /d "c:\DBRecovery\Mailbox Database 0436312751.edb"
eseutil /R E00 /l "c:\DBRecovery" /d "c:\DBRecovery\Mailbox Database 0436312751.edb"

It failed with an Error “-1216”. They did some Internet research and found that this could be fixed by using the “/i"

So their next try was:

eseutil /R E00 /l c:\DBRecovery /d "c:\DBRecovery\Mailbox Database 0436312751.edb" /i

eseutil /R E00 /l c:\DBRecovery /d "c:\DBRecovery\Mailbox Database 0436312751.edb" /i

And YES it results in “Operation completed successfully in 0.140 seconds.” Then they checked the Application event log and YES, there is no Error.

Application event log

Their last check before mounting the database was eseutil /mh "c:\DBRecovery\Mailbox Database 0436312751.edb."  After all that work – STILL a “dirty shutdown” Sad smile.

Note:

The get quick access to their data, they executed the eseutil command using /P for a repair to get the database into a clean shutdown. Microsoft strongly discourages using eseutil /P because it may lead to data loss. The /P should be a measure of extreme desperation when there’s no way at all to recover your database via /R.

Troubleshooting Steps – How did we solve it?

They showed me their eseutil command:

eseutil /R E00 /l "c:\DBRecovery" /d "c:\DBRecovery\Mailbox Database 0436312751.edb” /i

I saw three things I wouldn’t expect to see:

  1. “/i” is a parameter for skipping missing log stream attached databases. In older versions of Exchange one log stream was used per storage group. As a storage group could have contained multiple databases, the “/i” option was provided when only one out of multiple databases has to be recovered.   As Exchange 2010 doesn’t use storage groups any more, this parameter shouldn’t be used any more.
  2. “/d” parameter with the fully qualified path to the EDB file.  You HAVE to use the directory path and NOT the full path to the edb file.
  3. They also didn’t specify a system path, so the actual path of the cmd prompt is used. This directory is used to check for, or to create, an E0x.chk file. I always recommend specifying the same path as the edb and log directory to keep the files together, and ensure that the files you expect to be used are the ones which are always used.

First, I asked them to remove the “/i” parameter and to try it again, they told me that this resulted in the previously mentioned error:  “-1216 (JET_errAttachedDatabaseMismatch)”.

Checking the EventLog after removing the “/i” parameter, it stated that the database file has not been found at path “C:\DBRecovery\Mailbox Database 0436312751.edb\Mailbox Database 0436312751.edb" which makes sense.

Checking the EventLog

After removing the file name and ending “\”, and optionally adding the system directory parameter to the command, we ran eseutil /R E00 /l “c:\DBRecovery” /s “c:\DBRecovery” /d “c:\DBRecovery”.  Running this modified command, everything worked as expected and the database ended up in a “clean shutdown” without any error message.

eseutil /R E00 /l “c:\DBRecovery” /s “c:\DBRecovery” /d “c:\DBRecovery”.

Note: If the database still stays in “dirty shutdown”, check the specified system directory for an e0x.chk file from your previous attempts. This checkpoint file can be safely removed for this recovery process if only logs that are needed for your recovery exist in the log directory. Following that, the command should complete successfully.

So why was this happening?

I was wondering why two different customers would make the “same” mistake using eseutil /r, so I started some Internet research and found sites providing examples using “eseutil /r /d” with the full path to the edb. This will simply not work.  Most of these mention that sometimes this will not work – and recommend using /p to perform a repair of the database. As mentioned, this shouldn’t be the recommended way as long as the edb file and the required logs exist and are available to be used.

Another possible misunderstanding would be that eseutil also incorporates another “MODES OF OPERATION” which use “/d” as a parameter –> Database defrag. So if you use “eseutil /d” to defrag a database you have to specify a complete file path.

Hope this helps. And after reading through this, just sit back and use the ESEUtil parameter which you should try out with your Exchange 2007/2010 server (without any risk):  “eseutil /ESE.”  And give them their credits providing such a solid database Smile.

Thanks goes to Alexandre Costa and James Cameron reviewing this content.


Posted by Frank Battiston, MSPFE Editor

Comments (33)

  1. Anonymous says:

    Very Informative article on Exchange dirty shutdown which is a very common issue with Exchange server. Eseutil utility has innumerous commands to serve distinct need of the users. But if any case these inbuilt commands fails to bring in results then a third party tool always comes in use. Visit here: http://www.exchangerecover.com/recovery-tool-for-exchange.html

  2. Anonymous says:

    You know why this is happening? Because Microsoft Exchange, and all other pieces of software they ever made, are pure junk. It’s insane that some highly certified Microsoft certified person needs to write this kind of article, for all the windows sysadmins that even dare to touch the command line, and then encounter this kind of crap of some highly user unfriendly totally not working piece of junk tooling. I have tried to recover this so called robust database, and found the tooling does not work. The database keeps being “dirty” which is perfectly logical if you look at what this really is: junk that is a waste of sysadmins time. Anybody who uses Exchange or Outlook is simply insane for wanting to cope with this junk of complexity and error prone bad performing, non scaling, crashing piece of junk, that is not even possible to recover without 3rd party tools.
    It’s just rediculous. So anybody who reads this, be smart and do like us and migratie to a decent mail solution while you are dealing with this crap. Think Zarafa, or Google apps. Be amazed by the simplicity of backups and restores you can make with standaard tooling and mysql dump.
    Thanks for the attempt, too bad it failed for me. And probably for many others.
    Be smart, don’t use microsoft software if there is anyway to avoid it. Most other solutions give you more value for money, are cheaper to maintain and can actually make you tell your boss you are confident you can actually restore stuff when it is needed!

  3. Anonymous says:

    Thank you, thank you, thank you

  4. soniyaroy55 says:

    You can recover your deleted EDB file using EDB recovery tool from
    http://recoverydatatools.com/edb-to-pst-converter.html and convert EDB file to PST file format successfully. Read this post:
    http://edbtopstconvertertool.blogspot.com

  5. Alice.walker says:

    Nice information shared by author about why exchange databases might remain dirty after ESEUTIL /R recovery. It is a command line application that works with database files, ESE(Extensible Storage Engine ) and log files, which are associated with a MS
    Exchange database and repairs only mailbox and pulic folder of ESE. So you may take help this edb to pst tool (
    http://www.edbto-pst.com/ )to accurately recover exchange mailbox database with the complete details.

  6. Tuckor Alex says:

    It has been observed that in most of the corruption cases, ESEUTIL fails to repair the corrupt or damaged EDB file. It only works on those EDB files having minor corruption but for severe corruption, it is totally useless, even many Exchange MVPs say that.
    So, you can recover data from corrupt EDB file either by using the backup or by using any third party Exchange Recovery Software like Stellar Phoenix Mailbox Exchange Recovery (http://www.stellarinfo.com/edb-exchange-server-recovery.htm)
    . These third party recovery tools are capable enough to repair severely corrupt EDB files.

  7. Anonymous says:

    Great articles! Very informative !!! Thanks for clarifying, that was so ease to understand.

  8. Anonymous says:

    Great article! Easy to have customers follow the step by step instructions.

  9. specia01 says:

    Try Exchange database recovery software and successfully recover edb files and convert in outlook pst format.
    For more information free download click
    http://exchangedatabase-recovery.blogspot.com

  10. Anonymous says:

    Very informative blog to exchange user, i have repaired exchange server edb files by an effective third party software,that provides instant solution. For more detail you can see –
    http://www.edbtopsttutorial.net/

  11. Naveen - CSS says:

    Good one. Thanks for sharing this info.

  12. Andrew S Higginbotham says:

    Cool article Danijel. Also, I totally didn't know about the /ESE trick. Pretty cool & nerdy at the same time 😀

  13. Hi Daniel, very good write-up on this topic. I have encountered this scenarios several times.

    Thanks!

  14. Jennifer Lauren says:

    Wow!!!!  It is a very detailed description about how Exchange Server recovery can be possible, but in case if you need effective third party tool then you can go for Stellar Phoenix Exchange Server Recovery. I used it and it worked like a second life for my database.

  15. Kumararaja says:

    awesome :-)…

    Nice one to read and understand…

  16. Olive Pattionson says:

    Great Information for recovering Exchange database via ESEUTIL commands. However in case where Exchange Mailbox database is highly corrupted & Exchange server is even crashed then you can call for trusted Exchange Recovery solution ( http://www.pcvita.com/exchange-database-recovery.html ). ESEUTL will run only when Exchange server is live otherwise no way-out is left.

  17. Lastly, I had a unusable .ost file which was not working due to sudden server crash. I needed to recover that .ost file because some crucial information was in that file. I had searched lots of tools over internet but enable to find best one.<br />

    One of the user suggest me to use SysTools OST to PST converter software and i tried that many times and happily recovered all items of .OST file into PST file format.<br />

    People can search this tool directly here: <a href="http://www.microsoftosttopstconversion.scanostfile.com/…/www.microsoftosttopstconversion.scanostfile.com
    </a>

  18. shivenrosi says:

    It's really very informative article about Exchange server Recovery,If you want to know more about other option for Exchange Server Recovery, then I would like to suggest you for an professional Exchange server recovery tool like Kernel for Exchange Server recovery Tool. By using this effective software one can very easily repair EDB files of Exchange server without any hassle.

  19. Chac - SSE says:

    If we receive “-1216 (JET_errAttachedDatabaseMismatch)” best way to force them to replay the logs is mounting the database on Exchange server with required log files plus 15 – 20 log files in sequence. Just make sure you delete the Chk file before mounting it.

    Note : this is applicable only to Exchange 2007 and above. If we Still unable to mount its possibly page corruption which don't have other option than hard repair "eseutil /p"

  20. Abhishek says:

    Thanks .. it was really useful ..

  21. CarlJones says:

    Work with outlook conversion with the aid of ost to pst converter

    OST to PST Converter Toolbox works with ost and pst data

    Visit next resource http://www.osttopstconverter.convertertoolbox.com

  22. Steve Y says:

    I had been following the directions of the ‘other’ web sites that had the ‘eseutil /r… .edb’ and had the exact same problem you described. Your directions cleared things up. Thank you very, very much!

  23. P.C. Wren says:

    Thanks for sharing the useful information. Dirty shutdown is really a serious concern and sometimes leads to severe corruption in Exchange Server databases which can not be fixed via eseutil /r or eseutil /p utilities. In such situation, using a third party tool is a good option. One such popular application that helped us, can be downloaded from here: http://www.recover-computerdata.com/exchange-server-recovery.html

    A free trial version is also available on its official website !

    Wren

  24. Mark Clain says:

    Informative Article. But if in case it becomes difficult to recover Exchange Mailboxes with inbuilt utilities then one can opt for GUI bases application through which it becomes easily possible recover Exchange mailbox database.

    For more info:
    https://sites.google.com/site/exchangeedbrecoverysoftware/

  25. Eswar Maguda says:

    Excellent information…

    Could you please explain the usage of ESEUTIL /cc command while recovering a database from a online backup restore? I’m not so clear with respect to ESEUTIL /cc command as compared to ESUTIL / R and ESUTIL /p commands.

  26. Jack Brien says:

    Tips are really useful !
    Found an another excellent solution to handle Exchange dityshutdown problem in Exchange server:

    http://rickytechblog.wordpress.com/2012/08/17/tips-to-fix-exchange-dirty-shutdown-state-in-exchange-server/

    Hope these step by step solution will work to those who are in need !

    Jack
    Exchange Admin

  27. alex says:

    Eseutil is not useful in case of severe corruption of EDB file due to dirty shutdown. So, in that third party tools like Stellar Phoenix Mailbox Exchange Recovery , Kernel Exchange Recovery etc comes to the rescue. Third party exchange recovery tools are
    capable enough to repair any severely corrupt EDB file and restores the totally inaccessible mailboxes easily and swiftly. Find more info about the software at
    http://www.stellarinfo.com/edb-exchange-server-recovery.htm

  28. Petr says:

    The best article I have ever seen. Actually it fixes my problem. Thanks.

  29. Nathan Tuckey says:

    i’ve got the same issue, my database remains in a dirty shutdown

    Eseutil /r e02 /d "E:MailboxUser Store 2User Store 2.edb" /a /i

  30. Martin Miller says:

    Hi…..If this recovery solution doesn’t work then try "Exchange Server Recovery Software". It will solve all queries related to "Exchange Server Recovery". Try its DEMO Version to test its capabilities.

    For more info visit: – http://www.outlookemails.net/recovery/edb/

  31. Syrian Eagle says:

    Woow. You save us

  32. Syed says:

    I am facing the same problem, I have complete logs eseutil /r E00 command worked fine but still the Database is in the Dirty Shutdown

  33. Arwen says:

    Great article. Very helpful.