Outlook-Performance verbessern mit neuem Hotfix

Auf dem letzten Microsoft TechDay in Hamburg wurde ich noch gefragt, welche Maßnahmen man zur Verbesserung der Performance von Outlook 2007 bei der Nutzung von sehr großen Postfächern durchführen sollte. Bisher gab es dafür eine Reihe von Artikeln, die derartige Maßnahmen beschreiben:

Vor zwei Tagen stellte Microsoft nun kumulative Hotfixes für Office 2003 SP3 und Office 2007 öffentlich zur Verfügung. Neben die vielen Problemen, die mit dem Hotfix angegangen werden, stecken in dem Outlook 2007 Hotfix jede Menge Backports von Änderungen zur Verbesserung der Performance, die wir im Rahmen der Entwicklung des kommenden Microsoft Office 2007 Service Pack vorgenommen haben.

Generell verbessert der Hotfix für Outlook die Reaktionsgeschwindigkeit der Anwendung erheblich für Nutzer von Postfächern (OST/PST) im Gigabytebereich. Wir optimierten viele Lese- und Schreibzugriffe in eine OST/PST-Datei zur Verringerung der immer wieder ausgeführten Bedienschritte in Outlook (“day to day use”) notwendigen IO-Operationen. Durch die performanceoptimierte Ablage kann der generelle Platzverbrauch für die Datendateien um bis zu 20% steigen. Wir sehen aber mit den neuen Algorithmen drastische Zugewinne bei der Schreibperformance und eine generelle Reduzierung von notwendigen Schreibzugriffen auf die Festplatte.

Denjenigen, die PST-Dateien nicht lokal, sondern auf einem Netzwerkshare betreiben, sei an dieser Stelle noch einmal der sehr gute Artikel Network Stored PST files ... don't do it! des Performance-Teams ans Herz gelegt. In ihm wird im Detail erklärt, warum das keine gute Idee ist und deshalb auch nicht supported wird. Da die Liste der Änderungen in dem Hotfix so umfangreich sind, zitiere ich zum Schluss einmal den relevanten Teil der Beschreibung auf englisch:

Performance improvements

Performance and responsiveness are key concerns for all our customers. That is why we made the large performance tuning and optimization changes that are included in Office suite Service Pack 2 (SP2).

Outlook 2007 SP2 delivers performance improvements in four major areas:

  • General Responsiveness: SP2 reduces I/O disk usage and UI response time.
  • Startup: SP2 removes long operations from initial startup.
  • Shutdown: SP2 makes Outlook exit predictably despite pending activities.
  • Folder/View Switch: SP2 improves view rendering and folder switching.

Calendar improvements

Enterprise-class calendaring has evolved into a complex system of synchronized clients and devices. Customers have multiple versions of Outlook and Microsoft Exchange in their organizations, and they also have many mobile devices that have calendar access. Each method for accessing and updating user calendars typically offers some form of read/write permission back to the original calendar, frequently by using uniquely defined validation logic. In enterprise delegate manager scenarios, there is added complexity when many delegates are working on the same principal calendar at the same time. We recognize that customers require a high standard of reliability in these complex systems. Therefore, the Outlook team has improved the underlying data structures and general reliability of calendar updates.
This article includes a detailed summary of the recent fixes in the "Individual bugs that are fixed" section. This should give you a broad overview of the improvements in performance and calendar reliability and the other, more general improvements.

General improvements

In addition to the core goals of improved performance and improved calendar reliability, Outlook 2007 SP2 contains many fixes that, in aggregate, improve stability and also address customer issues. The most important of these fixes directly target the top known issues that cause Outlook to crash or become unresponsive. After you install SP2, you will notice increased stability over long periods of Outlook usage.
Other fixes in SP2 are spread across the following areas:

  • Data File Checks: Far fewer data file checks at startup, where you see the following alert:

    The data file '<file name>' was not closed properly. This file is being checked for problems.

  • Search reliability: Improved search reliability when you use SP2 with Windows Desktop Search 4

  • RSS: Fewer duplicated items

  • Security: General improvements in security

  • Object Model: Now contains many customer-driven fixes

Outlook 2007 SP2 performance

General responsiveness

The overall responsiveness for Outlook 2007 SP2 is significantly improved over the responsiveness of Service Pack 1. Responsiveness can be loosely defined as the application's reaction time when you use the user interface (UI). A perfectly responsive application will always immediately begin performing the task that is requested even if that task takes a long time. For example, in Outlook this could be clicking to switch folders while you wait for a very large folder to load, and clicking the ToDo Bar to enter a task.
Making Outlook noticeably more responsive was a big priority for this release.

Storage engine

The most important changes come from underlying optimizations in the local storage engine. Both Personal Folders files (PSTs) and Cached Exchange Stores (OSTs) are built upon the same underlying storage engine. In this article, PST refers to both technologies. We have improved the general reading and writing characteristics of the PST so that less is demanded from the disk to perform the same general operations in Outlook. For example, file flushing is optimized so that it interacts more efficiently with the operating system and disk write caches. We have also changed the way new data is written to the PST. Now, it is saved much closer together on the physical disk and requires less physical work from the drive. This speeds up the time that is required to write the same amount of data. A side effect of this data writing algorithm is that PSTs are approximately 20 percent larger than in previous releases. However, we have observed large gains in write performance and a reduction in overall writing to the disk.
Whereas write optimizations have been the single biggest change in the storage engine, there are also other small changes that deliver additional benefits. For example, internal storage engine optimizations were made after analyzing the most common kinds of operations the storage engine performed during typical usage. We have also optimized the way internal "change notifications" are processed, especially when there are many of them. Specifically, when you add significant data to the PST, lots of notifications are triggered for various data structures to be updated to accommodate the new data. Because these notifications are faster and require less I/O, the raw performance of every small operation improves. This means significant savings over time.

User interface

In addition to optimizing the storage engine, Microsoft spent time and resources investigating and addressing some common sources of unresponsiveness in the Outlook application layer. We have seen that some of the most frustrating instances of unresponsiveness occur when Outlook is doing work unrelated to any user actions. Such situations are especially frustrating because the user cannot predict when they will occur or how long they will last. Two of the more common causes of Outlook unresponsiveness that are unrelated to user action were processing updates to the message list and processing updates to other elements in the main window in Outlook. Our engineering team undertook a complex effort to fundamentally change the methods in which such updates are processed to minimize their effect on Outlook responsiveness in SP2.
Some other application layer changes that we made are for scenarios that may not affect all users. However, we believe that these scenarios are a significant source of dissatisfaction to the users who are affected. For example, when the reminders dialog box appears, the prior behavior was to process and write the changes to the store immediately when the user requested an operation, such as dismiss or snooze. Now, those changes are queued so that responsiveness is immediately restored to the UI. Therefore, you can click Dismiss as fast as your mouse will allow, and when the last reminder goes away, the dialog box will immediately disappear. We also made some changes to the way rules are processed so that many client-side actions, such as moving mail from a specific distribution list to a separate PST file, leave the Outlook UI far more responsive.
We have also made changes specifically targeted at customers who use POP and IMAP servers. For POP mail users, we now retrieve the newest mails first. This should help you get to your most important work faster. IMAP mail users will notice that downloading message bodies no longer stops Outlook from responding to additional user commands.

Starting Outlook

Our approach to optimizing the startup process is fairly straight-forward. We analyzed all the tasks Outlook queues and improved the boot time by removing a few operations that were considered unnecessary, and by better orchestrating the execution of the necessary tasks.
The result is a noticeable positive improvement. Quantifying the exact improvement is difficult because most people's experiences vary widely based on several factors. These include the following:

  • Number of add-ins
  • Amount of memory on the computer
  • Speed of the hard disk drive
  • Number of other applications currently running
  • Size of the mailbox and size of additional stores mounted (overall optimizations in the PST also help here)
  • Number of customizations made to the UI that must be loaded

Essentially, we have seen Outlook starting about two times faster in many of our tests and when we run builds for our daily mail use.

First startup experience

Immediately after you install the SP2 update, the first startup of Outlook will not be fast because Outlook must update internal PST data structures that are used in our optimizations. This is a one-time cost, however, and it affects just the initial startup. As soon as the data structures are updated, Outlook can then benefit from the improvements. We hope that, as soon as the upgrade process is complete, and you begin to use Outlook 2007 SP2, you see that the rewards were definitely worth the wait. There is a new upgrade dialog box with progress to show some optimizing work Outlook is doing to make everything faster on the disk.

The responsiveness of Outlook will improve after this dialog box closes. If you choose to create a new profile and resynchronize your mail, the storage upgrade happens dynamically as new content is added to your new local data file. Therefore, there is no upgrade dialog box. However, some non-responsiveness that is associated with the first startup may occur. Again, after several minutes, Outlook responsiveness will improve.

Shutdown

Outlook had built-in logic that caused it to refuse to shut down when it detected that a third-party was using it for data access or otherwise was integrated with Outlook functionality. A side effect of this design was that, to the user, Outlook would frequently fail to exit for no apparent reason. If all storage providers support it, we have optimized SP2 to shut Outlook down quickly by stopping all server activity, flushing unsaved PST content to the disk, saving unpersisted preferences and settings, and then quickly exiting. Most notably, Outlook no longer waits for external references to be released.

Note Outlook Live Connector (OLC) 12.1 does not support this faster shutdown. Therefore, Outlook must continue to use the slower shutdown method when OLC 12.1 is being used to make sure that data is not lost.

The result is a consistently reliable shutdown, even during periods of heavy activity, such as server synchronization. There is more technical information available for developers about how to update their applications to best integrate with this new shutdown model. Our goal is that Outlook shuts down quickly (in just a few seconds), reliably, and consistently. Also, a new Outlook icon appears in the notification area after you close Outlook if the Outlook.exe process is still running. As long as the Outlook.exe process is running, and the Outlook UI is unavailable, the following Outlook icon appears in the notification area:

For folder switching and arrangement switching, we have developed a highly optimized, very compact new index of the folders' contents that is sorted by date. A date sort is the foundation of the most frequently used contents table arrangement. Being able to load a very small and always up-to-date version of this index in a short time is the backbone to a very fast folder switch experience in SP2. The new date index is much more effective, even with tens of thousands of items in a single folder.
The result is that switching to even the largest folders takes seconds, although sometimes the first switch to the folder can take longer while we build up the indexes. This, again, is a one-time cost.

Note Most of these gains appear when you use Outlook in Cached Exchange Mode together with OSTs and PSTs. Online Mode users will continue to see their view performance dictated largely by server performance and network conditions.

Conclusion

Overall responsiveness has significantly improved with Outlook 2007 SP2. In our own daily usage of SP2 across several hundred computers, we have received much positive feedback regarding improved responsiveness.
Be aware that decreasing reading and writing demand on the disk has a positive effect on the minimum hardware necessary to maintain a good user experience. This is especially true for early generation solid state hard disk drives and for large local storage files that are frequently associated with large mailboxes or with large archive folders. To give customers more specific guidance, we are working on a more concrete numeric analysis of these improvements.
Outlook 2007 SP2 is a significant upgrade over Outlook 2007 SP1. Many of the most common tasks, such as shutdown and folder switching, are much more responsive in targeted scenarios.

Outlook 2007 SP2 calendaring

Because having the same data everywhere is foundational to a reliable calendar, it is no surprise that much of our work focused heavily on Cached Exchange synchronization. We improved the synchronization engine so Outlook automatically resolves conflicting edits without generating alternative copies of the items in the Conflicts folder. We have also turned on the conflict resolution engine for delegates who cache their manager's calendar. By focusing heavily on synchronization, we have mostly eliminated issues that caused problems such as the following:

  • Duplicate copies of a meeting
  • Organizer have a different meeting version than attendees
  • Disappearing meetings
  • Reappearing meetings (usually exceptions)
  • Meeting that are removed from the organizer but still remain on attendee calendars

General reliability

In addition to the synchronization focus, there are also a collection of general purpose reliability fixes that are important. For example, in Cached Exchange Mode before SP2, an edit to a meeting could sometimes generate an error message that indicates that the item has been changed and that your changes cannot be saved. SP2 addresses this situation and prevents the conflict from disallowing saving. If you used Outlook add-ins before SP2, regardless of how many times that you tried to change an item, the change was never saved until Outlook was restarted. This issue is addressed in SP2. Also, add-ins that use persistent calendar transactions should be more reliable.
We have made other reliability improvements, such as keeping private items private on updates, the ability to dismiss reminders, and the elimination of duplicate recipients. Additionally, some issues that are related to the prompt for sending only to added or removed attendees are fixed. These issues generated an update to everyone on the meeting.

Time zones

Time zone definition and daylight savings definition changes are becoming globally more frequent. In response to this increasing customer need, we have changed how we deliver updates to the time zone and daylight savings tool. With SP2, updates are removed from Outlook and provided exclusively in the Time Zone Data Update Tool for Microsoft Office Outlook . The download tool has also been updated to include Start menu entries for increased visibility. This tool is supported in Outlook 2000, Outlook 2002, Outlook 2003, and Outlook 2007.

Conclusion

We have fixed many underlying problems in calendar use that could have previously caused unpredictable behavior. We want to make sure that everyone's calendars contain consistent information and that appointments work smoothly. We understand the critical importance of reliable calendaring to our customers. Therefore, we have spent a lot of time capturing feedback and debugging problems to improve the calendar experience.

Individual bugs that are fixed

The following fixes are included in the February Cumulative Update:

  • If you click Remove from Calendar for a canceled instance of a recurring meeting, the whole series is deleted.
  • Synchronization conflict resolution frequently causes Exchange server conflict items.
  • When you close an e-mail message, Outlook 2007 crashes.
  • Attendees can cancel meetings on behalf of organizers in Outlook 2007.
  • Synchronization with SharePoint could generate error logs even when logging is not turned on.
  • After a series update, if an instance is moved to previously occupied timeslots, attendees may lose an instance.
  • Assume the following scenario:
    • An organizer sends a recurring meeting.
    • The organizer moves the instance to a different day.
    • Attendees accept the series and the update.
    • An attendee proposes a new time.
    • The organizer accepts the new time proposal and then sends the update to all attendees. In this scenario, the meeting time reverts to the original time that was proposed.
  • A local change to a meeting item is not synchronized to the server. Sometimes, the meeting disappears and is moved to the local failure folder.
  • When multiple processes are accessing the same data file by using MAPI, the Outlook UI does not receive the highest priority access to data files.
  • A reminder cannot be dismissed on a corrupted meeting.
  • When you build information to display new mail notification alerts, Outlook may periodically become unresponsive.
  • If a meeting is corrupted, an attendee becomes the organizer and can cancel the meeting.
  • After you add many feeds, Outlook crashes.
  • If you act on meeting requests before the Calendar folder is synchronized, the meeting response states are reverted. For example, the declined meetings come back. Or, meetings are duplicated.
  • If you update the status bar during Outlook synchronization or during a folder switch, Outlook becomes unresponsive.
  • When Outlook is synchronizing with multiple RSS feeds, Outlook crashes.
  • When you import lots of RSS feeds from an Outline Processor Markup Language (OPML) file, the CPU usage of Outlook is very high.
  • Some users may not change the recipient list when they reply to all recipients on a digital rights management (DRM) protected item in the Sent Items folder.
  • When you remove an attendee from a recurring series that has an updated instance, the attendee is not removed from the updated instance.
  • More logging capabilities are added when logging is turned on. This can help the support team investigate calendaring issues.
  • Downloading a large message blocks the user interface.
  • A solution is developed to process internal change notifications in the background. This enables the user interface to be more responsive to user input when significant background processing occurs, such as synchronization or indexing.
  • When you download CSS definitions while Outlook displays a message, the user interface becomes unresponsive.
  • Inefficient processing occurs during folder switch operations. Sometimes, the user interface becomes unresponsive.
  • When you work in Outlook 2007, and you enable the Show in Groups option, folder switching becomes slow, and a performance decrease occurs.
  • Users cannot select which pages to print of a message item.
  • Three or more processes access a MAPI store. If one process stops unexpectedly, Outlook 2007 becomes unresponsive.
  • You open an instance of a recurring series and edit it as an organizer. When you receive a response from a user for the series, your changes are saved without your interaction.
  • Licensing validation can cause Outlook 2007 to become unresponsive when it first starts after you install or upgrade Outlook.
  • If the Mailto hyperlink contains international characters, the hyperlink does not work.
  • If the store providers are disconnected early, the Outlook.exe process becomes unresponsive for a very long time.
  • The synchronization between Outlook 2007 and a SharePoint document library has a URL size limitation of 260 characters.
  • Assume the following scenario:
    • You run a search in an Outlook 2007 folder.
    • You run the search in All Mail Items.
    • You click the show all results bar. In this scenario, the bar does not disappear.
  • The synchronization between a calendar on a SharePoint Web site and Outlook causes Outlook to become unresponsive.
  • Assume the following scenario:
    • You run a search in an Outlook 2007 folder.
    • You run the search in All Mail Items.
    • You click the show all results bar. In this scenario, a timing issue occurs, and nothing is returned.
  • If a distribution list (DL) that includes a meeting organizer is removed from the meeting, the organizer receives a cancellation. The organizer can remove the meeting without sending a cancellation. Additionally, a meeting that does not have an organizer cannot be updated or canceled.
  • When Outlook 2007 restarts, it does not resend the AutoDiscover counter. The counter may not run for at least six hours.
  • When you save an Outlook 2007 message, it takes a long time. This problem occurs when you install InfoPath on your computer.
  • The Offline Address Book download starts immediately at startup. A change was made to start the download after initial mail synchronization for improved responsiveness.
  • When you roam SharePoint lists to a new Outlook profile, Outlook keeps adding duplicates of personal folders.
  • If a RSS feed item download is interrupted, the RSS feed items that are deleted by users reappear.
  • If a distribution list is part of the attendee's list, an attendee receives a cancellation unexpectedly. This problem occurs when the Send updates only to added or deleted attendees option is enabled.
  • When you copy lots of items in the same store, it takes a long time. This problem occurs when you have InfoPath installed on your computer.
  • When you issue a query to check the Out of Office settings on an Exchange server, Outlook becomes unresponsive immediately.
  • Opening mail items in a window or reading mail items in the reading pane may be slow if many messages are already open in other windows.
  • POP servers should download the newest mail first instead of the oldest mail.
  • Folder sort indexes are not saved in certain cases. This causes Outlook to recompute the index for each folder switch.
  • A reminder dismissal occurs as a background task. This change instantly restores UI responsiveness after a reminder.
  • A meeting that is copied from a shared calendar does not indicate that it is a copy. SP2 adds "Copy:" to the subject line.
  • Moving lots of items causes the UI to pause for a long time and high CPU usage to occur.
  • Date sorting is not efficient.
  • AutoDiscover in Outlook fails if the user domain names are long.
  • When you close an item, the clipboard is cleared unnecessarily. Outlook becomes unresponsive when the clipboard content was retrieved from slow applications or remote applications, such as a Terminal Services session.
  • Duplicate calendar entries, duplicate attendees, old-style conflicts because of using more than one client or running against Exchange Server 2007 or later.
  • Exceptions to a recurring meeting show duplicate attendees.
  • The Outlook calendar sometimes displays appointments and meetings in black text on a black background when you use high contrast mode.
  • Sometimes an update is sent to all attendees when you wanted to send an update only to added or updated attendees.
  • You receive a reminder for a canceled meeting, even though the meeting request was declined.
  • When you update the unread counts, Outlook becomes unresponsive. Additionally, the unread counts have been removed completely on operating systems, such as Windows Vista, that do not support Fast User Switching per user unread counts.
  • IMAP accounts become unresponsive during a Send/Receive operation.
  • Outlook 2007 becomes unresponsive shortly after startup because Outlook issues a remote procedure call (RPC) to the Exchange directory server.
  • When you run All Mail Items searches across multiple stores in which a result set can be returned, a blank view is substituted for the result set.
  • When you double-click an attachment, Outlook 2007 becomes unresponsive. This problem occurs when other applications on the system are not responding.
  • When you right-click an item, the whole item is loaded into memory more frequently than necessary.
  • Inefficient processing occurs in a loop during intermittent network connectivity.
  • Unnecessary disk reads are performed for every time that a custom form icon is rendered.
  • Administrators cannot use Group Policy settings to block the export of contacts from the global address list (GAL).
  • When you try to use the PropertyAccessor interface on a Recipient object to determine the Recipient flags, an error is returned.
  • Items in search folders sometimes disappear and do not reappear.
  • Outlook becomes unresponsive when you open an Information Rights Management (IRM) protected message.
  • After you discard some changes to a meeting that you own, an update for that meeting may be sent out when you close Outlook.
  • Data file checks occur too frequently. Also, data file checks to checkpoint and resume from a previous session is enabled.
  • If the organizer of a meeting is a member of an external distribution list and sends a meeting request to that external distribution list, the organizer becomes an attendee.
  • Bursts of write operations are flushed to the disk too frequently. This causes frequent pauses, and the UI becomes unresponsive.
  • Outlook does not add users to the publishing license when you use Reply to All from the Sent Items folder for new recipients.
  • Outlook ignores the IRM template. Therefore, a new license must be obtained every time that a protected message is opened.
  • If you change the file name when you save a file from an Outlook attachment, the extension is lost.
  • Outlook resets all favorite mail folders during a startup after a crash.
  • User-created add-in handlers for shortcut delete events ignore cancel commands, and the shortcut is deleted.
  • Form regions are sized to have 0 height or width when they are loaded. After the form regions are loaded, they are invisible.
  • The To-Do bar does not keep a minimized setting correctly, depending on each Outlook module.
  • IMAP accounts start slowly.
  • An entry for a Calendar or Contacts folder is duplicated in the Calendar or Contacts list, respectively. Only one of the duplicated links for a folder will work.
  • If you deploy the PSTNullFreeOnClose registry key, deleted data is not overwritten when you exit Outlook.
  • When you share a folder with a delegate, the delegate sees informational meeting updates for you as regular updates.
  • You receive the Application object instead of the Namespace object when you try to obtain the parent from an Accounts object.
  • Voice mail items display the sender's name in non-Unicode fonts.
  • The policy setting that enables or disables the RSS feeds synchronization between Internet Explorer 7 and Outlook does not prevent prompts.
  • RSS items are sometimes duplicated.
  • Outlook stops indexing mounted PSTs when a Classic Online mode account is used.
  • When you process client-side rules, such as moving mail to a PST, frequent pauses occur, or Outlook becomes unresponsive during synchronization.
  • A reminder cannot be dismissed or made to snooze. Therefore, a restart of Outlook may be required to dismiss the reminder or make the reminder snooze.
  • The new mail notification window unnecessarily reloads the delivered messages to check for additional and rarely used properties.
  • When you click a hyperlink, the user interface becomes unresponsive during a check of the Internet proxy server settings. These checks now occur after you click the hyperlink and in the background. Therefore, responsiveness is immediately restored to the UI.
  • When you enter double quotation marks in the Sent To field in a search folder, Outlook crashes.
  • RSS feeds become dormant and do not reactivate.
  • When the indexing process encounters problems, many event logs are generated, and CPU usage increases.
  • Outlook crashes when you create a message view.
  • Outlook re-adds itself to Indexing Options in Control Panel as a searched scope after you deselect it.
  • Server-relative links in SharePoint RSS feeds do not work in Outlook.
  • Outlook cannot search for items while in classic offline mode.
  • Outlook with Windows Search 4 stops indexing items if a PST is renamed in the Account Settings dialog box.
  • Unnecessary registry queries for ClearType checks occur during some UI drawing operations.
  • When you connect to an Exchange server over the Internet (RPC/HTTP) from a Windows Vista-based computer, you are repeatedly prompted for your credentials.