What is Central Management Store (CMS)

Update May 25, 2011: The official Server Administration chapter including the CMS section has now been published - https://blogs.technet.com/b/drrez/archive/2011/05/24/microsoft-lync-server-2010-resource-kit-server-administration.aspx

The following information is an official preview of a single section of the Microsoft Lync Server 2010 Resource Kit, which will debut chapter-by-chapter in 2011 on the DrRez blog and then later be available in its entirety. This text is © 2010 Microsoft and published with permission.

Central Management Store (CMS) is a repository to store management data in the form of topology, configurations and policies. The repository is implemented as an SQL database (xds[1]). The configuration information is stored as XML documents in the database. CMS services are built around this repository to replicate the management data to required machines.

The access to the repository is provided through and limited to a Microsoft Lync Server 2010 Dynamic Link Library (DLL) called Microsoft.Rtc.Management.Core.dll. The DLL also enforces validations and scope resolutions etc. All consumers of the repository and its data, including service components and Lync Server Management Shell, uses this dll to interact with the CMS store.

The way to modify information in CMS is by using one of the tools: Topology Builder (TB), Lync Server Management Shell (PS) and Lync Server Control Panel (LsCP). It is described in Figure 1.

Figure1

Figure 1 Interacting with CMS

CMS includes the key functionality of validating any information being written to it before it commits to the database. CMS implements and checks many rules for dependencies and inter-dependencies between components and server roles. In this way we make sure that the topology is valid and does not contain any un-supported configurations[2]. The validation logic is also available and used by the tools mentioned above.

CMS operates in a single master/multiple replica system. In every Lync Server 2010 deployment there is only a single master CMS and all servers running Lync Server 2010 have a local replica of CMS. All updates to information in CMS takes place at the master. The information is then replicated to all replicas. We will discuss how the replication works in the section How does CMS replicate.

It is recommended to install CMS on an Enterprise Edition (EE) pool so that it benefits from the high availability built into the Enterprise Edition. In such a configuration the master xds database is located in the same SQL Server instance as the backend databases, typically RTC. Each Front-End in the pool will have the replica xds database in a local SQL Express instance called RTCLOCAL. See Figure 2 showing a pool with 2 front-ends FE1 and FE2 and a SQL back-end BE.

Figure2

Figure 2 CMS database placement in an EE pool

If CMS is installed on a Standard Edition (SE) pool the master xds database will be located in the normal SQL Express instance called RTC. Mirroring the front-end servers in an Enterprise Edition pool the Standard Edition pool will also create a replica xds database located in the SQL Express instance called RTCLOCAL. See Figure 3 showing an SE pool SE.

Figure3

Figure 3 CMS database placement in an SE pool

The location of the master is found by looking up a Service Connection Point (SCP) for the master CMS. The SCP is an object in Active Directory created under the path of the following Distinguish Name (DN), CN=Topology Settings, CN=RTC Service,DC=<domain>, of type msRTCSIP-GlobalTopologySetting. This object contains the msRTCSIP-BackEndServer attribute, which specifies the FQDN of the master and the instance name of the SQL Instance. All tools use the SCP to locate and connect to the CMS master.

The SCP is typically created by Topology Builder when it initially publishes the topology based on information in the topology, and this is the recommended method for creating the SCP[3].

If you want to move the CMS store to a new pool the PowerShell cmdlet Move-CsManagementServer has to be used and it will also update the SCP to point to the new location.

In PowerShell the cmdlet Get-CsManagementConnection will show the connection information used when communicating with the CMS master. TB and LsCP will always read and write to CMS master. In PowerShell many of the Get-Cs* cmdlets accepts the switch –LocalStore, which will display the information found in the local replica of CMS.

The consumers, i.e. local Lync Server 2010 services running on the server, will always read from the local replica. This ensures resilience in case the master or the network connection to the master is not available. Resilience of CMS is discussed in more details in section Resilience.

CMS is implemented using three Windows services:

  • Lync Server Master Replicator Agent (MASTER)
  • Lync Server File Transfer Agent (FTA)
  • Lync Server Replica Replicator Agent (REPLICA).

All three services run on the CMS master; however, only REPLICA runs on the replica (that is every Lync Server). We will describe what the services do in the section on replication.

What is stored in CMS?

As described above CMS stores configuration information. In previous versions that kind of information was stored in Active Directory, SQL and WMI. So why was the choice taken to move it to CMS?

  • Storing the information in different places made it difficult for the administrator to understand where the information was stored and which interfaces should be used to manipulate it
  • When adding new features and functions to the product there was often a need to extend the Active Directory schema with new object classes and attributes. Extending the Active Directory schema is often a source for delay with customers
  • The location for storing information in Active Directory have changed between product releases and has led to issues for customers with global deployments using the deprecated locations
  • The Edge server role is typically deployed in the perimeter and therefore without access to the corporate Active Directory
  • Active Directory does not include rich capabilities for validation of dependencies, data validation, guarantee data consistency between replicas and the data model is not structured

The above reasons justified the need to create the central configuration information repository based on SQL. Lync Server 2010 still stores user type information in the user objects in Active Directory and maintains backward compatibility with system level settings stored in Active Directory for interoperability with Office Communications Server 2007 and Office Communications Server 2007 R2.

CMS stores information as XML documents. CMS stores three types of information: Topology, Policy and Configuration and these three types of information can be stored in four different scope levels: Global, Site, Service and Tag. The topology document contains the topology information generated by TB. The policy type contains all the different policies configurable in Lync Server 2010 (i.e. dial plan, client, conferencing etc.). The configuration type contains all the different settings, which can be configured in Lync Server 2010 (i.e. certificates, call park orbit number range, dial-in conferencing access numbers etc.).

There is only one XML document per information type and scope (i.e. one XML document will contain all dial plans with Site scope while another XML document will contain all dial plans with Global scope).

The tools will write to the XML documents through internal programmatic interfaces provided by CMS, i.e. Microsoft.Rtc.Management.Core.Dll. It is not supported to manipulate the XML documents directly.

How does CMS replicate?

As described above all writes are done to the CMS master and all Lync Server 2010 servers have a local replica of CMS. It is therefore necessary to update the replicas from the master. This process of transferring information updates from master to replica is called replication. The replication process consists of copying information between directories from the master to the replicas, applying the changes received to the replica and report status back to the master.

The CMS master uses a directory structure shared with other Lync Server components in network share defined in the topology document. The name of the top level directory is the Service Id of the CMS master. The tree structure is the following: <Lync Server FileStore>\<CMS Service Id>\CMSFileStore\xds-master, where <Lync Server FileStore> is the name of the directory selected to be the FileStore used by the CentralMgmt service in the topology document and CMS Service Id is the Service Id of the CentralMgmt service in the topology document . The directory, xds-master, contains two sub-directories replicas and working. The replicas directory contains a sub-directory for each of the replicas. Each sub-directory is named after the server’s FQDN. Within each replica direction are two sub-directories: from-replica and to-replica, (See Figure 4).

Figure4.JPG

Figure 4 CMS xds-master directory structure

Each replica uses a directory structure in the file share, xds-replica (\\<replica>\xds-replica), to synchronize with the CMS master, where <replica> is the FQDN of the replica. The xds-replica contains three sub-directories: from-master, to-master and working (see Figure 5).

Figure5

Figure 5 Replica directory structure

Each 60 seconds a task is run to determine if a change has been made to the CMS master and needs to be replicated. The unit of replication is the changed XML document, i.e. if a dial plan with global scope has been added, the full XML document containing dial plans with global scope will be replicated. All changes made since the last time the task was run are batched together into a data package – data.zip (see Figure 6). The data package also contains meta-data. (See Figure 7).

Figure6

Figure 6 Data package

The size of the data package is small, typically less than 100 Kb, which makes for fast replication across the network without any substantial impact to the bandwidth.

Figure7

Figure 7 The content of a data package

The MASTER generates the data package containing new changes to CMS and stores a copy in each to-replica directory for every replica.

The data package must be copied to all replicas. All Lync Server 2010 servers, except the Edge Server, uses the Windows file copy SMB[4] protocol mechanism to push the data package from the CMS master to the replica.

For Edge servers the file copy is performed over an HTTPS channel. The Edge Server runs a Web Service (https://<edgeserverfqdn>:Port/ReplicationWebService) on the port specified for the ManagementServices in the topology document. The default port is 4443. The Web Service, ReplicationWebService, does not require IIS to be installed on the Edge Server. It is implemented as part of the REPLICA service. The certificate used for the HTTPS channel is the internal or default certificate created for the Edge Server.

It is the responsibility of the FTA service running on the CMS master to copy the data packages from the master to the replicas. The FTA service has change notifications[5] on all the to-replica directories. It is alerted when the MASTER service places data packages in those directories and starts the file copy process to the replicas.

On a replica the REPLICA service has change notification on the from-master directory. It gets alerted when the FTA service has copied the data package to the replica. It then extracts the data package and applies the changes to the local CMS replica. After applying the CMS changes the REPLICA generates a status package – status.zip. The status package contains information for the CMS master about the status of the application of the CMS changes.

The REPLICA service will place the status package in the to-master directory. The FTA service running on the CMS Master has change notifications on the to-master directory on all replicas (except on Edge Servers). It is alerted about the new status package. On Edge Servers the FTA regularly polls the to-master directory to see if a status.zip package is placed there. Then polling frequency is increased when the FTA is expecting a status.zip package, i.e. after it has just transferred a data.zip package to the Edge Server. It then copies the status package from the replica to the from-replica directory on the CMS Master for the given replica.

The MASTER has change notifications on all from-replica directories. It will process the status package and update the CMS Master.

When a Lync Server starts up the REPLICA generates a status.zip package to report the current status to the CMS Master.

On the replica the Lync Server 2010 services (consumers) have registered for changes to the different types of XML documents in the replica. They will poll the local replica each 60 seconds to discover any changes to the XML document(s) they are interested in.

The two 60 second intervals means that it can take around 2 minutes from changes to be visible to the consumers (60 seconds poll on master + replication delay + 60 seconds poll on replica)

Resilience

Microsoft recommends installing CMS on an Enterprise Edition pool for resiliency of the CMS services. One of the front-end servers in the pool is designated as the active master. All front-end servers in a pool run the MASTER and FTA services, but only on the front-end server designated as the active Master are CMS tasks performed.

To ensure high availability, a heartbeat mechanism is used to failover the active master to another front-end server if the current designated front-end server is not responsive. To determine which front-end server is designated as the active master use the PowerShell command:

Get-CsManagementStoreReplicationStatus –CentralManagementStoreStatus

In addition to the high availability of the CMS services built-in to the front-end servers architecture of an Enterprise pool, the CMS database can be made resilient by using SQL Server Clustering. To protect the data stored in the CMS database, SQL Server backup procedures can be leveraged using the cmdlet Export-CsConfiguration –Filename <file.zip> on a regular basis.

As previously mentioned the services from Lync Server 2010 server will always communicate with the local CMS replica and not the CMS master. If communication between the CMS master and the replica is disconnected, then the replica fails to receive any CMS updates. Nevertheless, the Lync services are able to continue functioning even without the latest CMS updates. When communication is re-established the local replica updates and the Lync services obtain the latest CMS information. If the replica has been without contact with the CMS master for an extended time the CMS master will gradually bring it up-to-date. It might take up to 1 hour to get the replica fully updated.

If the CMS master is not accessible from PowerShell or Lync Server Control Panel, updates (add or remove information) in CMS is not possible..

Managing CMS replication

Managing CMS replication involves checking for replication status and invoking replication. Checking for replication status is done using the PowerShell command Get-CsManagementStoreReplicationStatus. This command shows information shown in Table 1 for each replica in the topology.

 

Item

Explanation

UpToDate

The replica is up-to-date with the CMS master

ReplicaFqdn

The full qualified domain name of the replica

LastStatusReport

latest date and time when the MASTER received a status package from the replica

LastUpdateCreation

latest date and time when the MASTER generated a data package to the replica

ProductVersion

version of Lync Server

Table 1 Output from Get-CsManagementStoreReplicationStatus

If Get-CsManagementStoreReplicationStatus is run with the parameter –CentralManagementStoreStatus it will display information about the active master. The information listed with this parameter is shown in Table 2.

Item

Explanation

LastUpdatedOn

The last time the master CMS database was changed

ActiveMasterFqdn

The full qualified domain name of the Lync Server designated as the active CMS master

ActiveMasterLastHeartBeat

latest date and time when the active master sent a heartbeat

ActiveFileTransferAgentFqdn

The full qualified domain name of the Lync Server designated as the active File Transfer Agent

ActiveFileTransferAgentLastHeartBeat

latest date and time when the active File Transfer Agent sent a heartbeat

ActiveReplicas

The list of active (not deleted) replicas

DeletedReplicas

The list of deleted replicas (servers no longer active in the topology)

Table 2 Output from Get-CsManagementStoreReplicationStatus -CentralManagementStoreStatus

To force replication you can use the PowerShell command Invoke-CsManagementStoreReplication. The command clears the last status report field from the CMS, which triggers replication. This command should be used with caution as is generates unnecessary traffic on the network due to the replication of all configuration to all machines.

Both Get-CsManagementStoreReplicationStatus and Invoke-CsManagementStoreReplication have the parameter –ReplicaFqdn, which will scope the action to only the specified replica.

Acknowledgements

I would like to express my gratitude to Erdinc Basci, Serkan Kutan and Santhosh Misro who designed and implemented CMS and tirelessly explained CMS to me. Thanks to Rui for his multiple reviews of the text and his help in making it more English Smile However all mistakes are my own.


[1] Xds is an abbreviation for XML Document Store.

[2] The OCS Best Practice Analyzer (BPA) provide configurations checks to flag such issues post deployment.

[3] It can be changed using the PowerShell cmdlet Set-CsConfigurationStoreLocation –SQLServerFqdn -SqlInstanceName

[4] https://msdn.microsoft.com/en-us/library/cc246231(v=PROT.13).aspx

[5] https://msdn.microsoft.com/en-us/library/aa365261(VS.85).aspx