After Installing Exchange 2010 Service Pack 1, Searching via OWA or Outlook Online Mode Fails


Update 7/2/11: The installation steps have been modified to make it clear the script must be executed on the Mailbox server using elevated credentials.

With Exchange 2010 Service Pack 1, you may find that performing a search via Outlook Web App (OWA) results in an error message, “The action couldn’t be completed.  Try again later.” Additionally, Outlook clients operating in Online Mode can no longer search for attachments or embedded emails. 

When this issue occurs, you will see the following events in the application log:

Log Name: Application
Source: MSExchangeIS Mailbox Store
Event ID: 9877
Task Category: Content Indexing
Level: Error
Description:
Content Indexing function ‘CISearch::EcGetRowsetAndAccessor’ received an unusual and unexpected error code from MSSearch. Mailbox Database: <Database Name> Error Code: 0x80043629

Log Name: Application
Source: MSExchangeIS Mailbox Store
Event ID: 9842
Task Category: Content Indexing
Level: Error
Description:
Function CISearch::EcGetRowsetAndAccessor detected that content indexing was disabled for database ‘<Database Name>’ because of error ‘0x80041820’ from MSSearch.

When you check the CatalogData folder in the same subfolder as the database, you may see that the content indexing files (.CI files) are not growing. If you perform a ResetSearchIndex.ps1 on the index, you may see that the total size of the CatalogData folder only grows to a few KB or a few MB and either no .CI files are generated or only a few .CI files are generated and are only a few KB each.

What causes this issue?

This problem typically is a result of installing or upgrading a multi-role server (a server that has the Mailbox, Client Access, and Hub Transport roles installed) to SP1 and then removing the Hub Transport and Client Access roles, thereby leaving the server with only the Mailbox role installed. 

There is a symbolic link for each language referenced in the registry,  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ContentIndex\Language.  The symbolic links (ActiveX CLSID of the class used) allow for word breaking and stemming  for the locale corresponding to the language.

During the Exchange 2010 Service Pack 1 uninstall process for the Hub Transport role, setup removes the symbolic links, regardless of the fact that the Mailbox role is still present.  The removal of the symbolic links causes the generation of content indexing files to fail.

How do we resolve this issue?

You will need to execute a script on the affected Mailbox servers that reinstalls the symbolic links:

  1. Download the script Repair-ExchangeSearchSymlinks.ps1 from the Script Center.
  2. Copy the Repair-ExchangeSearchSymlinks.ps1 script to the Scripts directory on the Mailbox server (default installation is C:\Program Files\Microsoft\Exchange Server\v14\Scripts).
  3. On the afflicted Mailbox server, open the Exchange Management Shell by right-clicking the shortcut and selecting “Run as Administrator”.
  4. Within the Exchange Management Shell, navigate to the Exchange Scripts directory.
  5. Within the Exchange Management Shell, run the following script:

    .\Repair-ExchangeSearchSymlinks.ps1

  6. Within the Exchange Management Shell, run the following script (without the symbols [] and substitute the database name(s) for any databases you wish to reset:

    .\ResetSearchIndex.ps1 [-force] <dbname> [<dbname>]…

  7. You will receive an Event ID 109 when the rebuilding of the index starts for each database and an Event ID 110 for each database when the index rebuild has completed.
  8. After receiving Event ID 110 for each database, test to make sure search functions correctly with both OWA and Outlook operating in Online Mode.

Bob Want


Comments (21)
  1. MCSE says:

    Is symbolic link  a UNIX/Linux term?

  2. HotFix says:

    Definitely a good one to know about if we change remove the server roles on an Exchange 2010 SP1 machine. Thanks for keeping these types of posts coming proactively.

  3. Ross Smith IV says:

    @MCSE – I've updated the article to provide more information about these symbolic links.  Thanks for helping us improve the article.

    @Hotfix – Thank you for the feedback.

    Ross

  4. Paul Glass says:

    I've upgraded two multi-role exchange servers to SP1 and all suffered this problem. No roles were removed or changed so I dont think its specific to doing an SP1 upgrade and then removing roles. This article has helped me fix it. Nice one, its been bugging me for months. I've tried to find a fix but have really struggled.

  5. Richard Sobey says:

    Hi Bob, Ross,

    *Looks back through the event log on one of my servers* I'm seeing these, 155 entries since 27/06/2011:

    Log Name:      Application

    Source:        MSExchangeIS Mailbox Store

    Date:          01/07/2011 10:14:06

    Event ID:      9877

    Task Category: Content Indexing

    Level:         Error

    Keywords:      Classic

    User:          N/A

    Computer:      <>

    Description:

    Content Indexing function 'CISearch::EcGetRowsetAndAccessor' received an unusual and unexpected error code from MSSearch.

    Mailbox Database: DB001

    Is this the same thing as you're describing? The server was installed SP1 and has never been multi-role. We're also not getting the 9842 events. But online search is very hit and miss in my experience. I don't rely on it and use cached mode instead :)

    Regards

    Richard

  6. Bob Want says:

    @Richard Sobey

    The only 2 known Event IDs on this issue are:

    Event ID 9877 w. error 0x80043629

    Event ID 9842 w. error 0x80041820

    I have seen some cases in which only 9877 w. error 0x80043629 appears.

    However, the key is that Event ID 9877 can have many errors associated with it.

    You do not mention the error code you are receiving with Event ID 9877 – we would need that to answer your question.

    @Paul Glass

    I have found that 95% of the time, removing a role has played a part.

    However, in 5% of cases, I have found that a role was not removed.  However, the key to the entire problem is that this issue is only known on Exchange 2010 SP1 servers – it has never been seen on Exchange 2010 with no service pack.   So, installing Exchange 2010 SP1 appears to be defintely related to the issue.

    Bob

  7. Abhi says:

    How does it impact the server performance, I have these errors reported in the app log

  8. VX1200 says:

    With Error Code: 0x80041606.

    ======================================

    Log Name:      Application

    Source:        MSExchangeIS Mailbox Store

    Date:          7/3/2011 5:36:53 PM

    Event ID:      9877

    Task Category: Content Indexing

    Level:         Error

    Keywords:      Classic

    User:          N/A

    Computer:      

    Description:

    Content Indexing function 'CISearch::EcGetRowsetAndAccessor' received an unusual and unexpected error code from MSSearch.

    Mailbox Database: DB001

    Error Code: 0x80041606

    ========================================

  9. Jonathan Splatt says:

    This issue still occurs if you use an internal email address as the search term.

  10. Ian says:

    I am still seeing these errors after running the instructions above. I've gone through these steps on several mailbox servers and the error still pops up.

  11. Bob Want [MSFT] says:

    @Abhi

    It should not affect performance at all. If you have the errors, then you should run the script.

    @VX1200

    0x80041606 is a different issue and not related to this issue.

    @Jonathan Splatt

    Have not heard of that.

    The key is to apply the script and see if the errors in the application log go away.

    @ Ian

    Questions for you Ian:

    1. Is your server a Mailbox role only server or an All-In-One Mailbox, CAS and Hub?

    2. What are the error codes you are receiving for 9877 events?

    3. Follow these steps:

    a. Right click on the command prompt and choose Run as Administrator.

    b. Go to this directory: C:Program FilesMicrosoftExchange ServerV14Bin (or whatever drive you have Exchange installed on)

    c. Type dir *brkr.dll > symlink.txt

    d. Copy and paste the results from symlink.txt to the screen and post here

  12. Jonathan Splatt says:

    Bob, it doesn’t seem to be related to searching by email address as i first thought but the above script has definitely not fixed this issue in our environment. We are running an all in one setup and get the following error

    Error Code: 0x80041606

    symlink.txt contents below

    Volume in drive C has no label.

    Volume Serial Number is D4C1-1158

    Directory of C:Program FilesMicrosoftExchange ServerV14Bin

    12/07/2011  11:18    <SYMLINK>      chsbrkr.dll [C:WindowsSystem32chsbrkr.dll]

    12/07/2011  11:18    <SYMLINK>      chtbrkr.dll [C:WindowsSystem32chtbrkr.dll]

    12/07/2011  11:18    <SYMLINK>      defbrkr.dll [C:WindowsSystem32naturallanguage6.dll]

    12/07/2011  11:18    <SYMLINK>      korwbrkr.dll [C:WindowsSystem32korwbrkr.dll]

    12/07/2011  11:18    <SYMLINK>      thawbrkr.dll [C:WindowsSystem32thawbrkr.dll]

                  5 File(s)              0 bytes

                  0 Dir(s)  23,521,230,848 bytes free

  13. hutch says:

    Found this whilst troubleshooting seeing this in the logs:

    Type: Error

    Source: MSExchangeIS Mailbox Store

    Event ID: 9877

    Event Time: 06/08/2011 12:47:08

    User: n/a

    Computer: ex01.domain.co.uk

    Description:

    Content Indexing function 'CISearch::EcGetRowsetAndAccessor' received an unusual and unexpected error code from MSSearch.

    Mailbox Database: Mailbox Database 3

    Error Code: 0x80041606

    This is on Exchange 2010 SP1 with RU 3v3, combined HT/MBT/CAS.

    Be very grateful if anyone know what causes it and what fixes it.

  14. Bob Want [MSFT] says:

    @Johnathan Splatt

    Jonathan, if you could give me your email address, I will contact you offline.

  15. Jonathan Splatt says:

    Bob my work address is

    j.splatt@tozers.co.uk

  16. lhansen@microdata.com says:

    I have a all-in-one setup (it's a SBS2011 box), and I'm seeing this same thing. I'm getting the two event IDs with the errors indicated in the article (9877 and 9842, with error 0x80043619). I followed the steps indicated, which appears to have caused the recreation of the index for the mail database. But, even after getting the event id 110  to indicate completion, the errors come back as soon as I attempt a search in OWA.

  17. Bob Want [MSFT] says:

    @ LMHansen_007

    Hi there, can you give me your email address and I will contact you offline.

    Bob

  18. mugundhan.aswin@hotmail.com says:

    We have a similar problem with the search catalog and so owa users are not able to search the mails. our Db's are in a DAG spread across 3 sites. We even deleted the search catalog and restarted the exchange search service , but no improvements. our setup is like this… win2k8 exch 2k10 sp1 with ru3.

  19. Daniel Russello says:

    We are running an all in one Exch2010 sp1 UR4-v2 on 2k8R2 SP1 an the above solution did not work for us either. Our problem started with the installation of SP1 for Exch2010. We can search fine in Outlook and in OWA "unread mail" but not in standard folders.  I followed the above instructions carefully and received all of the event log entries specified without error.

  20. Robert says:

    I'm seeing the same thing on an all-in-one setup (SBS 2011).  I've got SP1 RU3-v3 installed and have followed the procedure in this blog posting.  I verified that I saw event 109 at the start of the index rebuild process and event 110 when it completed.  However, even after following this process, I am still seeing the same errors.

    Search in OWA fails with the message "The action couldn't be completed.  Please try again." and causes event 9877 to be logged with an error code of 0x80043629.  Running the Test-ExchangeSearch cmdlet also generates event 9877 with an error code of 0x80043629.  The Test-ExchangeSearch cmdlet fails with an error of "Time out for test thread.".

    I am also seeing intermittent occurrences of event 9842 with an error code of 0x80041820.

  21. bhup18@gmail.com says:

    Has this script worked for anyone? I'm seeing multiple resolutions and all with mixed results across different threads.

    Any other updates on this Bob? Thanks for your time.

Comments are closed.