This was a question coming from an education customer in Minnesota deploying OCS R2 Group Chat Server:
Their Group Chat client was working but the Group Chat administrator console was not. It was getting this error:
“Cannot sign in because of a problem with the chat room service….”
Server 2 received error while subscribing to peer 1, <1> <net.tcp://ocsgroupchat.campus.xxx.edu:8011/MGC/PeerService> <ChannelServer>. Details: Identity check failed for outgoing message. The expected DNS identity of the remote endpoint was 'ocsgroupchat.campus.xxx.edu' but the remote endpoint provided DNS claim 'ocscontent.xxx.edu'. If this is a legitimate remote endpoint, you can fix the problem by explicitly specifying DNS identity 'ocscontent.xxx.edu' as the Identity property of EndpointAddress when creating channel proxy.
Here are some support steps I found to check on your Group Chat Server installation that can relate to this error:
1. Verify the certificate assigned to the group chat server it should have both Server and Client Authentication. If you have only server authentication, sign in to admin console will fail with above error.
2. Above service accounts must be part of RTCUniversalServerAdmins group and also Administrators group of the group chat server. Also add the user name (admin account) with which you are going to sign in to group chat.
3. Enable the admin account along with the above five service accounts for SIP communication on OCS 2007 server R2. Configure them for Federation,PIC,Remote User Access and Enhanced presence.
4. From SQL server management studio ->Security->Logins, Make sure all the above service accounts and admin account, are there. Then here Login properties, General-> default database for each of the account should be "GCDB", Under user mapping->check db_owner for all the service accounts.
5. Under GC admin sign in console->Edit Accounts Settings->Automatic
Configuration->Uncheck "Use my Windows credentials to log in automatically" , then under Office communications Server leave Host "blank", select encrypted radio button, under Group Chat Server Settings leave Use default server address box
unchecked and server address as "OCSchat service account uri"
6. C:\Documents and Settings\All Users\Application
Data\Microsoft\Crypto\RSA\MachineKeys re-add the Lookup Service account and the Channel Service account and give them full control to this folder and re-apply full control to all the files in it and try to restart the services.
In their case, Step 6 resolved the Group Chat Admin Console error above.
For more on what is OCS R2 Group Chat Server read my other post here.