Adjusting the MOSS ROBOTS meta tag for 3rd party search engines – using a Control Adapter (last update: April 11th)


When developing a public facing website using the publishing features of MOSS it might be required to emit a ROBOTS meta tag that prevents internet search engines from indexing specific pages.


The standard RobotsMetaTag control included in WSS generates the following tag:


<META NAME=”ROBOTS” CONTENT=”NOHTMLINDEX”>


Unfortunatelly most search engines do not understand the NOHTMLINDEX content value but a value of NOINDEX.


I already posted a solution for this which replaced the RobotsMetaTag control. Here is now a more elegant solution based on ASP.NET Control Adapters. Control Adapters can be used to modify the rendering of a control based on different browsers. But by using the “Default” browser it also allows to modify the default rendering of a control. Thanks to Gael Duhamel for the idea!


Here is the source code of the control adapter which we will use to modify the behavior of the RobotsMetaTag control:


using System;
using System.Collections.Generic;
using System.Text;
using System.Web;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.Adapters;
using Microsoft.SharePoint;

namespace StefanG.ControlAdapters
{
    public class RobotsMetaTagControlAdapter : WebControlAdapter
    {
        protected override void Render(System.Web.UI.HtmlTextWriter writer)
        {
            SPWeb web = SPContext.Current.Web;
            if (web.ASPXPageIndexed == false)
                writer.Write(“<META NAME=\”ROBOTS\” CONTENT=\”NOHTMLINDEX,NOINDEX\”/>”);
            else 
                writer.Write(“”);     // updated April 11th to fix a WebControls/Controls cast problem.
        }
    }
}

The code looks very similar to the code I used in my first approach.


To build the adapter you need to create a class library project, add this source code to it, configure the project to sign the assembly and add the resulting DLL to the global assembly cache (GAC).


To bind the control adapter to the RobotsMetaTag control we now need to add a new browser (e.g. SharePointAdapters.browser) file to the App_Browsers directory of your SharePoint web application.


The content of the browser file needs to look similar to this (you need to adjust the assembly name and strong name key based on your specific assembly):


<browsers>
  <browser refID=”Default>
    <controlAdapters>
      <adapter 
        controlType=”Microsoft.SharePoint.WebControls.RobotsMetaTag
        adapterType=”StefanG.ControlAdapters.RobotsMetaTagControlAdapter, RobotsMetaTagControlAdapter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=dcee90dea8c5fab5” 
      />
    </controlAdapters>
  </browser>
</browsers>

That’s it! After recycling your application pool the RobotsMetaTag control will now render it’s content based on the custom control adapter. No modification to your master pages will be required.

Comments (8)

  1. Kevin Mullins says:

    Thanks Stefan – have been strugling with this recently.  

  2. Anonymous says:

    Mashup Virtual Earth und SharePoint Wunder der Welt Suche Search Custom Security Trimmer for Novell Netware

  3. Seed says:

    Is there any problem in removing this tag? why is it there in the first place?

  4. Stefan Goßner says:

    Hi Seed,

    it is there to allow authors to decide if a specific site/page is allowed to be indexed by search engines or not.

    Cheers,

    Stefan

  5. Harmjan Greving says:

    When I add a (second) browser file besides compat.browser. I see no changes in my web application. SharePoint continues using the settings from only the compat.browser file.

    I would like to have the following browser files active:

    – App_Browserscompat.browser

    – App_BrowsersMyControlAdapters.browser

    Is this possibe?

  6. Stefan Goßner says:

    Hi Harmjan,

    that’s what I did.

    Cheers,

    Stefan

  7. Steven Van de Craen says:

    Harmjan (and others), if’ve just run into the same issue and the fix is to open the compat.browser in Notepad and save it (no need to change anything). This will force reloading the browser files. If you have multiple browser files already present you might need to repeat for each one.

    HTH

  8. Stefan Goßner says:

    Hi Steven,

    the RobotsMetaTag controls emits the settings configured in "Site Actions" – "Site Settings" – "Modify all Site Settings" – "Search Visibility".

    Cheers,

    Stefan