Many clients have wondered, why doesn’t ISA support defining multiple server certificates for a single IP. Such feature could have been useful when publishing several sites over SSL using the same public IP. On such configuration published site is using a different external names (e.g. mail.contoso.com, docs.contoso.com, …), where all public names are mapped to a single public IP.
If the listener on ISA is using a server certificate using name of one site (e.g. mail.contoso.com), clients that access docs.contoso.com will get error prompt from the browser. The common solution for avoiding this prompt is by using wildcard certificate (for the name “*.contoso.com”).
The reason such feature is not provided by ISA due to an inherent limitation of the SSL protocol:
When the client sends the "CLIENT HELLO" SSL message, the server is expected to send back a server certificate. However, the "CLIENT-HELLO" does not contain any indication to the name of the server that the client is interested in (this indication appears only in the Host header of the HTTP request, sent only after the SSL handshake have already been established). Server has no choice but to return a single server certificate per the (IP,Port) pair (a.k.a. listener), which is the only thing he "knows" before receiving the HTTP request.
Future versions of SSL protocol may support this. In case they do, ISA will probably leverage this support to allow multiple server certificates assigned to a single IP.
Note on ISA 2006:
The “multiple certificates per listener” feature in ISA 2006 is targeted in completing the 2006’s SSO (Single Sing On) experience. ISA 2006 provides SSO, when administrator uses with a single listener. E.g. administrator can configure two publishing rules for site1.contoso.com and site2.contoso.com assigned to the same web listener (with SSO domain: contoso.com), in a way, that will require user to authenticate only once.
However, since user might probably use SSL, the administrator must be able to return two different server certificates from the same listener. He (the administrator) will still have to use at least two IPs on that listener due to the issue described earlier in this blog entry.
Zvi Avidor, ISA Server Product Team.