How To Do SharePoint Collaboration in the Absence of Proper Governance, Part 2

  • Article

Written by Daniel Brunet, Senior Microsoft Premier Field Engineer. In my previous post, I explained the goal of this series of articles is to help SharePoint farm administrators make decisions when they have to maintain one or multiple SharePoint farms without governance, and touched on one of the most flexible and potentially challenging service offerings within SharePoint: Collaboration.   This is a continuation of that discussion, focused on the following four elements of Collaboration:

  • Security
  • Customization
  • Capacity Management for Collaboration
  • Backup and restore

Security

Should I USE AD or SharePoint Groups?

This is a common question. In most organizations, there are security policies in place for AD groups. In SharePoint, it is always good to use groups and the decision factor is not about technology but about service. There are no specific rules and it will depend on many factors. I will try to give you some guidelines but remember that they may not always work in your organization.

While AD groups are excellent for groups that are commonly used in the organization, smaller or sporadic groups for a site will be better serviced by SharePoint.

Factors too look at:

  • Application vs. Collaboration
  • Ad Hoc and project sites
  • AD group process and overhead
  • Existing AD group to be reused
  • AD Groups within SharePoint Groups
  • Nested AD Groups

When you create a collaboration environment, you will end up with hundreds of sites. Creating AD Groups for each site would cause a process overhead for the security team. Also, in some cases like we mentioned before, they will be self-sustained/created sites managed by the owner.

In the same environment, AD groups can be used to manage the farm administrators or a group that defines who can access that farm. In these cases, the scope is much larger and the maintenance much lower.

If we look at content management like a departmental site, AD groups make a lot of sense since the departmental group already exist and is already maintained.

In the case of an application or a portal, it is more the policies that will define the decision. At this point, AD or SharePoint does not make much difference. Maintenance will usually be low.

Some organizations will do a 1:1 match between an AD Group and a SharePoint group. They will add the AD Group to a SharePoint group instead of assigning the permission level directly. The only benefit in this scenario is to add someone individually in that SharePoint Group without the need to request a change from AD.

Note that nested AD groups can have some limitations if there are too many levels.

Site Collection Administration

This is a tough decision. If you put yourself as the site collection administrator for all those sites out there, you will keep some level of control but will surely cause an overhead in your administration work. Instead of just managing the farm, you will also manage site collection settings. Also some features may be required for an owner to manage.

The exception to this is sites with publishing features like portals. Some caching options are available that may have an impact on your web servers. Site collection administrators are highly controlled in a portal, as opposed to collaboration scenarios where it can be given to any owner.

Site Collection AdministrationSite Collection Administration

 

Note that there are two type of site collection administrators.  The ones created in Central Admin and limited to two and the ones added to the site themselves.

For the administrators added in Central Admin, they will have the following added responsibilities:

  • Receive Request Access emails
  • Get quota notifications
  • Get site expiration notifications

Custom permission levels

First let’s differentiate access rights and permissions. When you give access to a site, library or document, you need to identify what the actions are that this person or group can do against these objects. These actions are the permissions. A permission level or permission set is a predefined group of actions. So you give access to the site and at the same time need to apply permissions.

By default, SharePoint offers SharePoint groups that give you access to the root site and that match with a permission level.

Default SharePoint groups.

 

If we take Members that have the Contribute level, here are some actions that a user or AD Group in that group can do:

Some actions that a user or AD Group can do.

 

From here, you can create a custom permission level with appropriate actions and create a custom SharePoint group to associate it with.

A few times I’ve seen people giving Full Control or putting someone as a member of the owner group because the contribute level was not sufficient.

Let’s say you want some designer to have the ability to create a sub-site, but you do not want to give them full control. You should create a copy of the Design permission level and add the proper permissions there. You then associate a custom group or that specific user to the new permission level. If you need all designers to have this right, you can edit the default Design level instead.

Site Permissions

When you add rights to a user or an AD group, you have the option to make it member of a SharePoint group or to associate it to the permission level directly.

Note that this applies only to one site collection. Code is required to have this replicated to all site collections.

Sub Site Creation / Manage Permissions

If you are a contributor or part of the default Member group, you cannot create a sub-site or manage permissions. In some organizations, site owners will not have that option and will not be part of the Owner Group or have the full control permission level.

In these organizations, that functionality is blocked to prevent the owner to create sites that do not belong in that site collection or control any security. Of course, this will be more overhead for your team as you will have to manage sub-site creation on top of site collection creation and deal with many security requests.

For sub-sites, if you use proper quotas and the owner is well informed on what the service offers in your collaboration application, you can give that permission without any worries. It is part of what people like about SharePoint. It is more of an issue when people start to create sub-sites in applications like a portal or an intranet where it should be highly managed. Imagine your 2 GB intranet just bloated to 15GB because someone decided to create a departmental document storage repository in a sub-site.

For Security management in SharePoint Collaboration it should be the site owner that does this. If the site is a project site, the team that works on that project is not an enterprise scope but really a group scope. Small ad-hoc sites also have a very small security scope.  Normally, farm admins and security teams do not need to be involved in these sites.

Unlike Collaboration, a good example of blocking security would be the root of an intranet where people should not give rights to all the sections. But at a level down, the owner of a section or department could decide who the authors of the content are. At this level, it could be ok to give security management permissions.

Customization

There are many level of customization with SharePoint and I see many different expectations from my customers. Here are some examples I see often in Collaboration:

  • Requirement to use SharePoint without coding projects
  • Expected Change Management and versioning capabilities in SharePoint Designer
  • Plan to stage site collection deliveries
  • Use one site collection only so branding and customization does not need to be pushed by solutions and only by SharePoint Designer
  • Complete blockage of SharePoint Designer when a customer wants/needs it
  • No scripting capabilities in the team
  • No development team available for .NET, no solution built
  • Farm admins do not use/learn SharePoint Designer
  • SharePoint Designer is unknown or misused due to improper training
  • Composite Applications in the Collaboration environment

Composite Applications

SharePoint CapabilitiesOne of the great feature of SharePoint is its ability to build rapid applications.  These fall under the Composites application portion of the SharePoint Services Pie graph.  Your decision to let your organization create these in your Collaboration environment is completely at your discretion.  Composite applications are isolated in site collections so, in theory, there are no problems running these in the same web application as your project or team sites.

In my experience, the traffic will be different as you can have enterprise wide composite applications as well as departmental ones.  Sometimes, adding 3rd-party components to composite applications will bring a new layer of supportability and the associated complexity.  I’ve seen issues arise with very simple applications that should not have interfered with normal operations.

Here are a few examples of issues I’ve seen:

  • Looping workflows
  • Extremely large lists including the workflow history list
  • Too many list lookups
  • 3rd party component issues affecting server performance
  • Forms and columns which are too complex that should have used InfoPath instead
  • Absence of an archiving solution.  Items keep on piling up without retention and quickly reach number or size problems.

With SharePoint 2010, we now have better protection against these with list and web application throttles.  The addition of Sandbox solutions also helps in the decisions about customization.  But any design team should be aware of the boundaries and think ahead about retention and archiving.

So, back to your decision, what should you do?  For me, a simple move is to have a web application for each service.  Let’s say I handle project and team sites.  I could have one or two web applications for these.  For Composite Applications, I would have at least another one.  This way, I can isolate the worker process if I need to (One Application pool for applications, One for the Team and Project sites).

Remember that one limitation is 10 Application Pools per server and will also vary based on your available physical memory and user traffic.  This limitation is not for Web Applications as you can share many in one Application Pool.

Now I can also isolate which assemblies are offered in which Web Applications.  As well, I have the capability of enabling SharePoint Designer on the Composite Web Application while disabling it on the basic team/project ones if this is what I want.

Browser customization

In rapid development, many customizations will be done from the browser.  The typical customization will be custom site structure and site templates, lists to gather data and OOTB (out-of-the-box) workflows.  Many of these designers are not aware of the lists and views boundaries.  Fortunately, SharePoint 2010 protects itself but it does not help the team that may have to restructure its data.  Awareness is always good and the questionnaire should have helped already.

SharePoint Designer

First of all, I think SharePoint Designer is a great management tool.  Yes, it is an application to change the look of a site or do more complex page design, but it is just as useful for administrators as designers.  In my mind, every Farm Admin should have SharePoint Designer installed and should have a basic understanding of its capabilities.  If not used for your own benefit, at least you can look at what the designers do with it.

If a workflow misbehaves in one composite application, you can also have an insight on what is going on in that site.  Remember that SharePoint designer always work against a site collection.  It does not push anything to the servers and cannot be used to do mass customization on multiple site collections.

So what can we do with it?

  • Build custom sequential (now reusable in 2010) workflows
  • Manage Libraries and items
  • Edit Pages, Master, Page Layout (Publishing Sites) and Style Sheet
  • Customize Web Parts using XSLT
  • Connect to back End Data with BCS
  • Edit list forms (More complex forms are better handled with InfoPath)
  • and more…

Limitations

SharePoint Designer is good for working on one site.  When you do branding solutions, you can start your design with it, but you will need to move to SharePoint solutions and feature stapling when you need to apply this to large number of site collections.  Do not use only one site collection because of this.  It would be a bad decision in Collaboration and one that I saw too often.  I can guarantee you that you will break it down eventually.

SharePoint Designer does not offer versioning or staging.  If you designed a site collection and then copy it to production for release, you cannot do this a second time.  I am asked often how to apply changes with Designer outside of production and push them back.  It would require you to put your site collection to read only mode for a long period of time, copy it to a staging area and overwrite it to production after.  This does not make much sense in many scenarios.

If you created a set of lists, content types and items without a SharePoint Solution and Visual Studio, the only two way to update them is to manually apply the modifications directly in the production collection with Designer or to build a script that you can re-test many times in a copied test site and then run it against the production.  You can look at it like a SQL custom database where you need SQL statements to apply updates to the schema after release.  You do not overwrite a database every time you update it.  In the case of SharePoint, a simple console application or PowerShell can be used against the Object API to do your updates.

If instead you used features and code to activate your custom items, code will also be used to update these and will also support versioning of the deliveries.

Collaboration customization requirements

Usually, the typical customization will be fairly limited and pushed using a SharePoint Solution designed with Visual Studio and SharePoint Designer to a certain extent.  If you do not have any .NET developers, all you can do is site templates that are now saved as SharePoint Solutions from the Browser and available for new site collections.  While this is okay for some, it is fairly limited.

Typical customizations:

  • Branding
  • Enterprise Metadata, if required
  • Search Customization
  • Custom security at site creation
  • Custom self-provisioning site collection

So do you need development for your Collaboration? It always depends on the complexity of what is required.

Capacity Management

We cannot do Collaboration without capacity management.  One of my customers had a growth of 170% last year and SharePoint was not officially launched.  This tells you how much SharePoint is well adopted.

Boundaries and content databases

Your first task is to read and understand this article:  SharePoint Server 2010 capacity management: Software boundaries and limits.  Like I mentioned before, Collaboration is not content management.  By nature, sites should be fairly small in nature and have a life cycle.  Content management like that used for departmental sites will have much larger space requirements and life cycle management will be done on documents and libraries instead of the sites themselves. If your collaboration sites are permanent, then proper information management of the documents is necessary like in regular content management sites.

So in terms of calculations, your quota is the primary factor.  In general, not every site will consume all of it.  In fact, only a minority of sites will reach it and ask for more space.

It is important to understand that most content size boundaries are related to database and site restore operations.

Based on the boundaries provided, we can do some planning. Let’s say you have a 5 GB quota and want to keep your databases within the 200 GB recommendation.  That gives you a minimum of 40 site collections per content databases.  Because they will not all use their quota, it will in fact be more than this.  It is important to monitor the growth so you can switch to a new database when required.

If you read the TechNet article referenced above, you now understand the database size recommendations.  Since my post mainly focuses on out-of-the-box functionality, collaboration and no governance, we will stick to the 200 GB limit.

So you have a couple of hundred site collections in let’s say 10 content databases which are mainly under 5 GB in size each.  Some of these required an extension and you have one that is 20 GB in size.  Depending if you are running SharePoint 2010 or 2007 you may already face an issue.

Let’s say someone broke their site and is requesting a restore. Because you have a site collection for them, you know that you’re in good shape.  Since this site collection is shared with others in the same database, it is out of question to restore the database.  That would not please some of your customers.

Two things needs to happen.  First you need a copy of the database restored.  It can be on another SQL instance or the same one as long as you use a different name.  Second, you need to extract that site collection from the copy of the database.  This requires a SharePoint site collection backup operation.  If you are lucky and run SharePoint 2010, you can do this with the Offline Database feature.  It means that you can connect to that database to do your extraction operation.  If you are running 2007, you need a recovery farm (which can be a simple virtual single server farm with the same version where you can attach that copy of the database) to extract the site.

Extraction is done with STSADM with SharePoint 2007 or also PowerShell or Central Admin with SharePoint 2010.

Now that you have a backup of the site collection from the restored database, you can restore it to your production environment with stsadm or PowerShell.

Any gotchas?  Yes, one.  If you are running MOSS or WSS 3, the stsadm backup or restore command may fail when the site collection is larger than 10-15 GB.  This is a major issue when it is shared with other site collections in the same database.  If you run that version of SharePoint and a site collection is expected to be bigger than 10 GB, it should be isolated in its own database.  This way, if you need to restore the site collection, it will be done with a SQL operation overwriting the production database.

So what about SharePoint 2010?  The limit is now 100 GB.  If a site collection is to exceed that size in content management scenarios, it should be isolated in its own database.

Also remember that all of these boundaries are related to your SLA.  If a database or site collection backup restore operation takes too long, it will affect your expected recovery time.

rbs (SharePoint REMOTE BLOB STORAGE)

What about RBS, does it help in the capacity?  You have to know that the limits are the same. RBS is great for archiving and storage of large items like video libraries.  Collaboration requires many updates to documents and may be less efficient in RBS than in the database directly.  Actually, with RBS enabled on a database, smaller documents will be saved in the database anyway.

RBS is an abstraction on where BLOBs are stored.  SQL Server uses the FileStream.  Other vendors offer different methods.  When using RBS, you have to do proper planning and understand the implication in your backup/restore strategies.

Archiving

Every organization should have a retention and archiving policies.  Collaboration can be a bit different since the site may already have a short life cycle.  If not, it should comply with the Information Management policy of your organization.

You need to know what happens to the site once it is not used anymore or the project is completed.  Does it need to be archived and if so, what is the retention period before deletion?  Also in some cases, the project/team documents may need to be migrated to a more permanent site like a departmental site or intranet.  Since we don’t have a governance, at least it should be part of the site creation questionnaire so you have awareness.

Backup and Restore

We already touched a bit on backup and restore in our discussion about capacity management.  I see a lot of disconnect between enterprise backup strategies, the wish list of the SharePoint admins, and actual operations.

SharePoint has many levels for backup and restore.  Also, some vendors offer alternate solutions including Microsoft with its own product, Microsoft DPM.  I find it important that an admin understands how it is done with SharePoint and SQL only so you visualize every component.  Once you understand the concepts, you are better prepared to deal with an enterprise solution.

Levels

  • Server
    • OS
    • IIS Settings
    • Certificates
    • Customization
    • 3rd party
  • Farm
    • Settings
    • Web Application
    • Service applications
  • Content database
    • Site Collection
    • Sub site
    • Lists and libraries
    • Items and documents

Many of these components cannot be restored and must be rebuilt.  Also, in the case of disaster, you have to be able to rebuild your farm.  When I am delivering dedicated support engineering (DSE) to an organization, part of my work is to insure that the team can recover in any scenario.  Here are the items I will cover with the organization:

  • Review and validate the build documentation
  • Review and validate the customization deployment guides
  • Review and validate the 3rd party deployment guides
  • Review the database backup processes
  • Review if all sources, packages and keys are available
  • Evaluate the team knowledge on the different levels including a complete farm failure
  • Have the team execute a complete rebuild and review the process
  • Review the team process on content restoration

I will not go into all the details, but will mention that Microsoft Premier Field Engineering has an excellent 4-day workshop on this topic. Suffice to say that Microsoft does not support restoring the configuration database and that nothing beats good documentation and a prepared team.

Content restoration

In SharePoint Collaboration, you will mainly deal with content restore.  So if your SQL backup process is working well, you are in good shape.

Like I said many times, when you restore a site collection, you have a full fidelity restore.  When you start to restore only sub sites (and now with SharePoint 2010, lists and libraries) you are in fact importing content.  Some functionality may stop working.  Examples are workflows, list lookups, recurring meetings, etc.

Recycle Bin

Another new feature with SharePoint 2010 Service Pack 1 is the site recycle bin.  It’s a great feature to remember.  Talking about the recycle bin, it is your first line of defense.

Misconception number 1: the retention period applies to BOTH recycle bin stages.  By default, any items in the user and the admin recycle bins older than 30 days will be deleted.  The second stage does not give you another 30 days, it is only there so an admin can recover something removed from the user recycle bin.

Another common mistake is to disable the recycle bin timer job.  This will make your site collection reach its quota faster.  This should be part of your planning.

The user recycle bin goes against the site collection quota and the second stage adds 10% on top of the quota size.  A 10 GB site collection will add another 1 GB in the content database for the second stage.

Site collection orphans

A common mistake I also see is in the duplication of site collections.  Let’s say that your organization asks you to restore a site collection but would like to have a copy first.  In good faith, you will restore that site collection with a different URL.  But if you restored that collection in the same content database: too bad, that collection is now an orphan and you will have a GUID conflict.  The only way to do this is to restore that site in a different content database.  I tell my customers to have a separate web application for those staging collections so you can insure that you will not use the same content database.

Another way to create site collection orphans is by restoring a content database copy in the same farm.

Another good example is when you create a web application with a root site collection.  You then restore your own content database without detaching the other content database.  Your old root site collection won’t be accessible.

Required space

When you do a site collection restore, you need to do a SharePoint backup against a copy of the database. So that means free space.

On SQL SERVER

At a minimum, equivalent of the largest content database that can be copied for extraction operation.

On SharePoint Server where THE extraction operation is done

Equivalent of the largest site collection times two that you have in your farm.  When you run an stsadm backup operation against the copy of the database, a temp file will be used on the system drive so you need that much space.  As well, the actual result of the site collection backup size is required, most probably on your data drive.

Sub site, libraries and items

When you need to recover content that is not the complete site collection, you have some more steps to do.

First, you do the same as with a site collection by restoring a copy of the content database.  Again, if you have SharePoint 2010, you are in luck, no need to attach that database to a recovery farm and you can actually extract the sub-site or library directly from Central Admin.  Even better, you can import a library which was not possible with MOSS.  Once you have the exported packaged file, you then import it back to the site.  You can choose to overwrite the site or put it in an alternate location where the owner will do operations.  In the case of a single document, you will need to restore the library in an alternate location so the owner can recover the document.  Do not export at the item level.

If you are running MOSS or WSS, you have a couple more steps to do.  First, attach the copy of the database to the recovery farm.  Run an stsadm Export command to extract the sub-site to recover or the sub-site that holds the library to recover.

Import the sub-site to overwrite the site to recover or import it to an alternate location in the site collection so that the owner can copy the content.  Since you cannot recover or copy a library, each item will need to be copied back to the original library one at a time.

This concludes the Collaboration portion of this series.  Hope you found it helpful.