Setting up TMG 2010 Where EMS is a Domain Member and Array Servers are in a Workgroup

 

Introduction

I have seen a number of cases where customers were installing TMG 2010 in a “hybrid” scenario. What I mean by this is that the EMS was part of the Domain but the Array Servers were in a workgroup. There are a couple of “gotchas” that I wanted to talk about today.

Assumptions

I am going to make a few of assumptions before I get started. First I am going to assume that you have already installed the TMG Enterprise Management Server (EMS) on a server that is a domain member. I am also going to assume that you have installed the TMG Array Member on a server that is in a workgroup. I recommend getting both of them to the latest Service Pack, Updates, and hotfixes before proceeding. You definitely want both the EMS and TMG Array server to be the same code level. Please refer here for version number information on TMG.

Certificates and Accounts

The first thing you want to do is request a Server Authentication certificate for your EMS. It needs to be issued to the Fully Qualified Domain Name (FQDN) of the EMS with the Private Key Exportable option checked. In my lab the EMS is called ems.fabrikam.com so I requested a Server Authentication certificate from my Certification Authority and installed it into the Certificate Store. Make sure you also export the .PFX file for the certificate, with the private key, and put the file somewhere handy on the EMS machine.

Next you will want to make sure that both the EMS and the TMG Array Server trust the CA that issued the Server Authentication certificate to the EMS. You can do this by importing the certificate for the CA into the Trusted Root Certificate Authorities branch in the Computer Store on each of those machines.

Another thing that is sometimes overlooked in the scenario is that mirrored accounts are needed on both the EMS and the TMG Array Server. For example, I just used the fabrikam/administrator account on the EMS and the local Administrator account on the TMG Array server. They both have the same password.

Keep in mind that if you have any firewalls that reside between the EMS and the TMG Array Servers you will initially want to allow ALL traffic between them. You can tweak this down later but it can cause you a lot of heartache with communications in TMG.

Create the New Array

On your EMS, open the TMG MMC, highlight the Arrays branch, and then on the far right-hand Tasks pane choose to Create New Array

Fig1

Give your new array a name.

Fig2

Type in the DNS name of the array.

Fig3

Choose the Default Policy.

Fig4

Click Next at the Array Policy Rule Types.

Fig5

Complete the New Array Wizard.

Fig6

TMG will create the new array and you should see that it was a success.

Fig7

Apply this on the EMS.

Fig8

Wait for the configuration changes to be saved.

Fig9

In the TMG MMC on your EMS, there should now be a branch called Arrays. Below it should be the array that you just created.

 

Fig10

Joining the New Array

Back on your TMG Array Server go into the TMG MMC and highlight the branch that says Forefront TMG (servername). On the far right-hand pane under the Tasks Tab, click Join Array.

Fig11

You will see a welcome screen for the Join Array Wizard.

Fig12

Under the Array Membership Type choose to “Join an array managed by an EMS Server”.

Fig13

Give it the Fully Qualified Domain Name of your EMS. (Note: you will want to make sure name resolution is working properly on the TMG Array Server before you do this step).

Fig14

The newly created array should come up as a choice.

Fig15

Click Finish on the Completing the Join Array Wizard.

Fig16

You should get a message that you successfully joined the array.

Fig17

Give it a few minutes but you will probably notice that the configuration is not synching and you will get an error and a red X under the Configuration Status. The error reads “Forefront TMG Management cannot establish a connection with the Forefront TMG Computer.”

Fig18

The Problem

So why isn’t the TMG Array Server able to communicate with the EMS? It seems like everything was set up correctly. TMG in a workgroup scenario relies on Authentication over SSL encrypted channel (LDAPS). That is the reason we requested the Server Authentication certificate for the EMS Server.

You can verify this by going into the MMC on your EMS, right-clicking on the top level branch of the array that you just created and choosing Properties.

Fig19

Under the Configuration Storage branch the authentication type near the bottom should be set to “Authentication over SSL encrypted channel”

Fig20

The problem is that the Server Authentication certificate was never bound to the ISASTGCTRL service.

You can verify this by creating a Certificates Snap-in MMC.

Fig21

Choose to manage snap-in for a Service Account.

Fig22

Select the Local Computer.

Fig23

Select the ISASTGCTRL service and finish.

Fig24

You should now see that there is not a certificate under the Personal branch.

Fig25

Keep this open, you will refresh it in a few minutes.

To correct the certificate issue, download the TMG Cert Tool Pack from here.

Install the tool on the EMS but then move the ISACertTool.exe to the same directory where TMG is installed. Open an administrative command prompt and navigate to that directory. Run the command as explained below against the .PFX file you have for your EMS Server Authentication Certificate.

The syntax is listed in the DOC file that comes with it and is:

ISACertTool /st file_name [ /pswd password] [ /keepcerts]

Where:

/st file_name installs the exported certificate on the Configuration Storage server. File_name specifies the path and name of the exported .pfx certificate file.

/pswd password specifies the password that may be required when installing the server certificate. It is only required if a password was specified during export of the certificate file.

/keepcerts specifies that existing certificates should not be deleted. By default when you run ISACertTool.exe, all certificates in the ADAM_ISSTGCTRL local store are erased. To specify that existing certificates should not be deleted, specify the /keepcerts parameter.

After running this you should see a message that the Storage certificate installation was successful.

Fig26

Go back to the Certificate Snap-in for the ISASTGCTRL Service and refresh the Personal Branch. You should now see the Server Authentication Certificate.

Fig27

Now go back to Monitoring on your EMS and you should see that the TMG Server is successfully synching.

Fig28

Conclusion

Setting TMG up where the EMS is in a domain and the TMG Array Servers are in a workgroup can be tricky. TMG in a workgroup relies on an SSL Encrypted channel and sometimes getting that to work correctly is not always straight-forward. In this article I have shown you a couple of the common pitfalls and how to correct them.

 

Author

Keith Abulton – Security Support Escalation Engineer, Microsoft CTS Forefront Security Edge Team