Make head for headers

In a previous post, I discussed how UAG’s AppWrap and SRA mechanisms can help you solve problems and enhance functionality by performing on-the-fly modification of content that UAG processes. A lesser-known ability of AppWrap and SRA is editing HTTP Headers.

HTTP headers are data that web servers send and receive from web clients as part of an HTTP transaction. For example, when a browser sends a request for a web page to a web server, it adds a header that specifies what type of browser it is, and what are its capabilities. This header is the User-Agent header, and it could look like this:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.2; MS-RTC EA 2; MS-RTC LM 8)

Other HTTP headers in a request could include:

· The Accept header
· The Accept-Encoding header
· The Accept-Language header
· The Cache-Control header
· The Connection header
· The Content-Type header
· The Cookie header
· The Host header
· The If-Modified-Since header
· The Referrer header

When a web server replied to a request, it also includes some HTTP headers, which could include:

· Content-Encoding
· Content-Length
· Content-Type
· Date
· Expires
· Last-Modified
· Server
· Set-Cookie
· X-Powered-By
· X-UA-Compatible
· Connection

In addition to the above common headers, there could be custom headers, which could be anything you, or an application developer would like. For example, the popular finance application PeopleSoft adds the custom headers pscache-control, pscache-excludeparams, pscache-handler and many others.

The headers themselves have many uses. Some only serve to inform, like the Server header, which tells the client what is the server he is connecting to, and what version it’s running (for example, Microsoft-IIS/7.5). Others tell the browser what to do, such as the Content-type and Expires. Content-type tells the browser the incoming data is a web page, or possibly a special file that it needs to handle differently (like opening a PDF file with the PDF reader). Expires tell the browser when to purge the file from its cache and retrieve a new version of it from the server. The headers might be created by the server, or by the client, and both can also read the headers and make decisions based on what they see. For example, IIS uses the Host header to tell to which of several web-sites it might be hosting the request pertains to (this allows IIS to host multiple websites using just one server, one IP and one TCP/IP port).

One of the most important headers is the Cookie header. The Cookie is a special header, which web applications use to store data for various purposes. What’s unique about them is that the browser stores the cookies in a special place (usually on the hard drive) and if set in a certain way, could remain there after the client finishes using the application and closes the browser. This could be used, for example, by a shopping website, to store what items you looked at recently, and use that to offer them to you again upon your next visit. Another popular use is to track sessions – an application generates a unique cookie that remains on the client computer as part of the session, so upon subsequent requests for pages, the cookie is used to keep the session alive. In fact, that’s part of how UAG manages its own* sessions, as well as many other web applications.

Here’s a screenshot of a typical list of headers when visiting an application via UAG. First, we see the request headers, and below we see the response headers:

As you can see, each header is comprised of the header name, and its value. The Cookie header in the request is not identical to the Set-Cookie header in the response, even though they are related. Cookie is the header name for when a client sends the cookie (or cookies) it has to the web server, and Set-Cookie is sent by a web server when it pushes new cookies to the client. Cookies can also be edited on the client itself, using JavaScript and other client-side programming languages.

Editing headers with UAG

As I said in the beginning, UAG has the ability to modify headers on-the-fly. It can modify the headers when the go from the browser to the web-server (Request), or on the way back(Response). UAG can do this either using the application wrapper (AppWrap) or using the SecureRemoteAccess (SRA, also known as the AAP) engines. As you can imagine, there are 3 things you might want to do with a header – ADD one, Delete one, or EDIT one. You can do this with any header, including cookies!

AppWrap or SRA?

Although there are some similarities between SRA and AppWrap, their abilities in the context of manipulating headers are very different. SRA, for example, can perform search-and-replace on the content of headers, but it cannot ADD or DELETE a header. AppWrap, on the other hand, can add or delete headers directly.

On the other hand, SRA is much better at manipulating cookies, specifically. AppWrap can still do it, but it doesn’t have the means of manipulating the various parameters of the cookie that SRA can, such as the cookie’s domain and path.

Manipulating headers

Lets start by discussing simple header (not cookie) manipulation. As I just said, AppWrap is better at it, and here’s an example of such a manipulation:

The above is for ADDING a header named Content-Type, with the value of ‘text/html; charset=csISOLatinHebrew’, which would tell the browser to display the page using the Hebrew character set.

The example above is for the RESPONSE direction, so UAG would inject this into any page UAG delivers from the backend-server to the client. If you want the header to be injected on the way from the client to the server, use the REQUEST keyword instead.

The example above uses the SERVER_NAME structure to tell UAG on which responses to perform this. The purpose of this is to allow you to control this with a fine tooth comb. Typically, such a change would only apply to certain applications you are publishing, but not all. You can specify a server name using a shortname, an IP or an FQDN, but it should match the server you specified in the application properties. For example, if you want this to apply to your SharePoint pages only, and you listed the SharePoint’s server name as a shortname “SPSHaifa01”, you will have to match it in the XML:

The port is optional – you can specify it, or leave it blank, or skip the <port></port> line altogether. Another optional setting is the server mask, which is useful when specifying the server by IP. If you’d like to have this rule cover an entire subnet of server, you could specify something like:

<SERVER_NAME mask=”255.255.255.0”>192.168.1.1</SERVER_NAME>.

An alternative to this would be to use the APPLICATION_TYPE format. This is suitable if your application uses multiple servers, but not an entire subnet. You typically set the application type when you create an application, as part of page 2 of the wizard. :

For the built-in application templates, such as SharePoint, Lync, OWA and CRM, the type is preset in the template. The common application types are for built-in templates are:

· OWA2007
· OWA2010
· SharePoint2007AAM
· SharePoint14AAM
· Lync2010

For a full list of all types, open the following file:

<UAG Path>\von\Conf\WizardDefaults\WizardDefaultParam.Ini

To use the application-type format, you simply replace the <SERVER_NAME> line with:

<APPLICATION_TYPE>SharePoint14AAM</APPLICATION_TYPE>

Similar to other AppWrap work, the value of the cookie may need to contain characters that are illegal in XML files, and in that case, we can use Base64 encoding to make the file safe. To do so, use an encoding tool, such as the UAG Editor (from <UAG Path>\common\bin\editor.exe), or an online encoding website such as this. With encoded data, the value line would look like this:

<VALUE encoding=”base64”> Q3VyaW91cywgYXJlIHdlPyE/IT8h</VALUE>

Editing and deleting headers

The syntax for editing or deleting a header is pretty straight-forward. When deleting a header, there’s no need to specify the value, so it would be something like this:

In the example above, I also used a RegEx expression in the URL. RegEx can be used in URLs, header names, server names…but not in the application type.

Editing a header is simply based on a search-and-replace, similar to how you would do for a page’s content. Instead of enclosing your stuff in <DELETE></DELETE> or <ADD></ADD>, simply enclose it in <EDIT></EDIT> . For example:

If you do plan on using AppWrap to manipulate cookies, despite my suggestion to use SRA for it, keep in mind that the formatting for cookies is set-cookie when the cookie comes from the server (response), but just cookie when being set by the client (request). For example:

Manipulating cookies with SRA

As I said earlier, AppWrap is capable of some cookie manipulation, but its sibling SRA is much better at it, so we’ll turn to SRA to discuss that. The syntax of SRA is somewhat different. The thing that makes SRA more suitable is the fact that cookies, even though they are a type of header, have a big difference: a regular header has a name and value, but a cookie has a name, a value, but also several other properties:

· Path
· Domain
· Expiry date and time
· The HTTPOnly flag
· The Secure flag

Another property of cookies is its source. A cookie can be generated on the server, or on the client (with JavaScript, usually).

Here’s a sample of a simple SRA configuration to delete a cookie:

The above simply tells UAG to perform this on every request pertaining to the server Exchange01, and for such requests, remove any cookie named JSONPERS. A similar syntax can remove specific parameters or flags of the cookie:

This time, we aren’t removing the cookie itself, but the DOMAIN and PATH attributes of it. A variation of this is to remove a specific piece of data. For example:

<Domain remove="true">localhost</Domain>

<Domain remove="true">WHL_SERVER_NAME</Domain>

As you can see, the 2^nd example uses the constant WHL_SERVER_NAME, which tells SRA to look for a domain attribute matching UAG’s own server name and remove it. This can be useful in case you’re using this SRA on different UAG servers which have different names.

Other attributes you can edit using SRA are:

· Comment
· CommentURL
· Discard
· Max-Age
· Path
· Port
· Secure
· Version
· Expires

With these, you can ADD a value, or remove it, using the following structures:

· Remove any path: <Path remove="true"></Path>
· Remove a specific path: <Path remove="true">/path/</Path>
· Add a specific path: <Path remove="">/home/</Path>
· Add a specific path: <Path>/home/</Path>

As you can gather from the 3^rd and 4^th example, the REMOVE attribute can be empty if you’re not removing anything, and it can be omitted completely.

Preventing cookie signing with SRA

One thing that UAG does as part of publishing application is cookie signing. This process involves changing a cookie’s name to a unique string, such as I’ve shown in the beginning of the guide:

The purpose of this process is similar to server name signing (HAT – Host Address Translation). It allows UAG to match cookies in incoming requests to the backend application server they pertain to. For example, if UAG is publishing several applications running on different IIS websites, each of them would attach an ASPSESSION cookie to pages the server delivers to UAG. UAG needs to have a way of knowing which ASPSESSION cookie to send back to which server, because if it sent a wrong one (or all of them), that would confuse the session tracking mechanism on the backend IIS servers…and could lead to some major application issues. UAG encodes the relevant backend server info (hostname, port etc) alongside the original cookie name, and generates a unique name for it, which is how the client receives it.

One challenge with this is that if the client runs code that expects the applications’ cookie or cookies to be there, it could run into problems. Two applications that have this are Citrix and PeopleSoft, which have JavaScript code that goes into an error loop. For such applications, we need to configure UAG to NOT perform the normal cookie signing.

I’ve already published a post describing this with regards to PeopleSoft, but to repeat the idea here briefly, the concept here is that by telling UAG to make a minor modification to the cookie, we essentially prevent it from performing the signing process. For example, part of the custom SRA for PeopleSoft is this:

This tells UAG to remove the domain and path from the cookie. We don’t actually need the path or domain removed, but because of this, UAG never signs the cookie, which makes it fully available to the PeopleSoft code, and prevents the issue that would otherwise occur. Similar code can be used to have UAG avoid signing cookies for other applications.