Archiving Options in Lync Server 2013

Update 7/11/13 - Added "How Do I View Archived Content?" section

Lync Server 2013 has made a couple of changes to what can be archived and where that archived data can be stored.

 

What's New?

  • Collocation of Archiving on Front End Servers
  • Exchange Server 2013 Integration
  • SQL Store Mirroring
  • Archiving of Whiteboard and Polls

With the collocation of the Archiving Server role onto the Front End Servers, we've eliminated the need for a separate Archiving Server in Lync Server 2013.  This will help to reduce the server footprint required.  The Archiving service uses Unified Data Collection Agents that are responsible for capturing data and are located on all Front End Servers and Standard Edition Servers.  This also means that MSMQ is no longer needed!  It has been replaces by LySS (Lync Storage Service).  Also, you now have the choice to use Lync for the archiving store or Exchange Server 2013.  However, you can't archive to both locations at the same time.  Archiving to Lync is similar to previous versions.  The archived information is stored in the LcsLog database in SQL.  If you decide to use Exchange Server 2013 are the archiving store, you will be using OAuth and EWS to deliver the archived data to Exchange Server 2013.  Archiving also supports SQL mirroring for high availability and the database can be collocated in the same instance as the Front End Server databases, or it can be in a separate instance.

 

What Information Can Be Archived?

The following types of content can be archived:

  • Peer-to-peer instant messages
  • Conferences (meetings), which are multiparty instant messages
  • Conference content, including uploaded content (for example, handouts) and event-related content (for example, joining, leaving, uploading sharing, and changes in visibility)
  • Whiteboards and polls shared during a conference

The following types of content are not archived:

  • Peer-to-peer file transfers
  • Audio/video for peer-to-peer instant messages and conferences
  • Desktop and application sharing for peer-to-peer instant messages and conferences

The Lync Server 2013 Archiving service does not archive Persistent Chat conversations. To archive Persistent Chat conversations, you must enable and configure the compliance service, which is a component that can be deployed with the Lync Server 2013 Persistent Chat Server.

 

How Archiving to Lync Works

The first thing is to enable archiving.  You can do that in the Lync Server 2013 Control Panel or via the Lync Server Management Console:

EnableArchiving
Indicates which items (if any) are saved to the archiving database. Valid values are:

  • None. Archiving is disabled. This is the default value. If EnableArchiving is set to None then nothing will be archived in either your Lync Server archiving database or in Exchange 2013.
  • ImOnly. Only instant message transcripts are archived. If Exchange archiving is enabled these transcripts will be archived in Exchange 2013. If Exchange archiving is disabled then these transcripts will be archived to Lync Server.
  • ImAndWebConf. Both instant message transcripts and Web conferencing transcripts are archived. If Exchange archiving is enabled these transcripts will be archived in Exchange 2013. If Exchange archiving is disabled then these transcripts will be archived to Lync Server.

 

Once archiving is enabled and an Archiving Policy is applied to your users, the Archiving Data Collection Agent on each of the Front End Servers in the pool will monitor for IMs and/or Web Conferences that need to be archived.  It will send those to be archived communications to the LySS.  The LySS will then write it to the appropriate repository, either SQL or Exchange Server 2013.

 

How Archiving to Exchange Server 2013 Works

The process for archiving to Exchange Server 2013 works similarly to archiving to SQL.  When configuring the archiving settings for Exchange Server 2013, remember to check the box for Exchange Server integration.

EnableExchangeArchiving
When set to True, Lync Server 2013 instant message and conferencing transcripts are stored in Microsoft Exchange Server 2013 rather than a separate SQL Server database.  Note that if you enable Exchange archiving then users will be managed by the Exchange archiving policies instead of Lync Server 2013 archiving policies.  The default value is False.

Lync Server 2013 will use OAuth to connect to the user's mailbox in Exchange Server 2013.  You will need to configure both Lync Server 2013 and Exchange Server 2013 for Oauth.  You can refer to the Configure OAuth Authentication TechNet article for Exchange Server 2013 and the Managing Server-to-Server Authentication (Oauth) and Partner Applications TechNet article for Lync Server 2013.  For users with a mailbox on Exchange Server 2013 and on In-Place hold the Exchange Archiving Policies override Lync Server archiving policies.  You can also set the archiving policy manually for individual users using the Set-CsUser to select the archiving policy:

ExchangeArchivingPolicy
Indicates where the user's instant messaging sessions are archived. Allowed values are:

  • Uninitialized - Indicates that archiving will be based on the In-Place Hold settings configured for the user's Exchange mailbox; if In-Place Hold has not been enabled on the user's mailbox then the user will have his or her messaging and Web Conferencing transcripts archived in Lync Server.
  • UseLyncArchivingPolicy - Indicates that the user's instant messaging and Web conferencing transcripts should be archived in Lync Server rather than in Exchange.
  • NoArchiving - Indicates that the user's instant messaging and Web conferencing transcripts should not be archived at all. Note that this setting overrides any Lync Server archiving policies assigned to the user.
  • ArchivingToExchange - Indicates that the user's instant messaging and Web conferencing transcripts should be archived to Exchange regardless of the In-Place Hold settings that have (or have not) been assigned to the user's mailbox. 

 

Where in Exchange Server 2013 are Archived Lync Messages Stored?

When the user is set to archive into Exchange Server 2013, the messages are stored in a hidden folder (Recoverable Items > Purges) in their mailbox.  One way to verify this is to use EWSEditor:

Opening up the Purges folder will show you all the conversations that have been archived for that user.

 

Best Practices

One thing to check when setting up archiving to either SQL or Exchange Server 2013 is to run the Get-CsArchivingConfiguration cmdlet make sure that ArchiveDuplicateMessages is set to True:

From the Set-CsArchivingConfiguration TechNet article:

The ArchiveDuplicateMessages parameter specifies how "cross-pool" instant messages should be archived. Consider a simple example: Ken Myer (with an account in Pool 1) sends an instant message to Pilar Ackerman, who has an account in Pool 2. Pilar, in turn, sends a reply to Ken’s instant message. If ArchiveDuplicateMessages is set to False, then (based on a built-in algorithm) the session transcript will be logged in either Pool 1 or Pool 2, but not both. If ArchiveDuplicateMessages is set to True (the default value), the transcript will be logged in both pools.

Another parameter to look at and possibly adjust depending on your company policies is CachePurgingInterval.  From the Set-CsArchivingConfiguration TechNet article:

This parameter indicates how often (in hours) the system is purged of transcripts where none of the participants have been enabled for archiving. By design, all group IM sessions and conferencing sessions are recorded when they take place. At the specified interval, the system determines whether any of the participants in these sessions have been enabled for archiving. If the system finds a session where none of the participants have been enabled for archiving, then that transcript will be deleted from the database. The CachePurgingInterval property can be set to any integer value between 4 and 168, inclusive. The default value is 24.

 

How Do I View Archived Content?

One way to view archived data in SQL you can use the Export-CsArchivingData cmdlet.  This will export P2P and conference data to .eml messages.

This will export the conversations to the output folder and create folders for each date that a matching conversation was found:

If you open one of the folders you will see all of the conversations listed.  The file names have conf or p2p at the end so that you can quickly tell conferences apart from 1:1 conversations.  The conversations are stored as .eml files.  This means that you can open them using Outlook.

Opening one of the messages using Outlook will show you the transcript of that conversation:

 

Another way is to use a third party tool or report to pull information out of the archiving database.  I have created a Sample Lync Server Archiving Report that you can use:

You can find more information on the report, including the download link, here.