Content Deployment – The complete Guide – Part 4 – Communication

Content Deployment uses the http (or https) protocol for communication between source and target farm. Most of the communication is performed through Web service calls except for the file transfer of the exported files to the target farms which is done using a regular http POST request against an ASPX page to perform the upload operation.

Communication inside the same farm (between worker process and timer service) is implemented by writing to and reading from the configuration database (starting and stopping of timer jobs) and central administration content database (job status information) of the farm.

Communication through a Web Service

This Web service is hosted within the central administration Web application at the following URL:

http://target-central-admin-url/_vti_adm/ContentDeploymentRemoteImport.asmx

It provides the following operations:

GetVirtualServersInformation

This operation is used during the configuration of a content deployment path. It allows enumerating all SharePoint Web applications on the target farm which will then be presented in the UI to allow a user to select it as target for the content deployment path.

GetSiteCollectionNames

This operation is also used during configuration of a content deployment path. It allows enumerating all site collections on the target farm for a given virtual server which will then be presented in the UI to allow a user to select it as a target for the content deployment path.

GetRemoteAdminServerUrl

When configuring content deployment in a farm you have to define one server that has to act as the importing server for content deployment operations. The GetRemoteAdminServerUrl operation returns the information about this server to the caller.

This operation is called as well during execution of a content deployment job to identify which server in the target farm will perform the import.

CreateJob

This operation is called by an executing content deployment job on the source farm. It allows the creation of the shadow job on the target farm that will perform the import operation.

RunJob

This operation is called by an executing content deployment job on the source farm to start an earlier created shadow content deployment job on the target farm. This job will perform the import operation.

GetJobStatus

This operation is called by an executing content deployment job on the source farm in a configurable interval (default: 10 seconds) to get the status of the import operation from the target farm.

It should be noted that refreshing the content deployment status page does not necessarily result in a request for status to the target farm.

It should be noted that if the returned status does not change for a configurable timeframe (default: 10 minutes) the content deployment job on the source farm will report a timeout – independent whether or not the import operation later finishes successfully.

This affects mainly full deployment jobs which deploy a large amount of data. In this situation the decompressing phase on the target server can take longer than 10 minutes which will result in such a timeout.

CancelJob

This operation is called by an executing content deployment job on the source farm to cancel its shadow content deployment job on the target farm when a user requested the cancellation of the content deployment through the UI. Be aware that such a cancel operation can only be performed if the decompressing operation has not yet started.

DeleteJob

This operation is called by an executing content deployment job on the source farm to delete its shadow content deployment job on the target farm after the deployment has been completed.

File transfer through http POST

File transfer of the export packages to the target farm is performed through an http POST request to the following page which is hosted within the central administration Web application:

http://target-central-admin-url/_admin/Content%20Deployment/DeploymentUpload.aspx

The uploaded file is sent as payload of the POST request. The information about the name of the file and the shadow content deployment job the uploaded file belongs to is provided through query string parameters. The actual URL sent with the http request would look similar to this:

http://target-central-admin-url/_admin/Content Deployment/DeploymentUpload.aspx?filename=”ExportedFiles.cab”&remoteJobId=”49bebd7d-62d0-4d68-a1f0-9118b3ac4416″

The details of the web service and the file upload is discussed in detail in the following protocol specification: [MS-CDEPLOY]: Content Deployment Remote Import Web Service Protocol Specification

Content Deployment Communication Flow

To describe process used during content deployment we will use following diagram. Be aware that communication in the same farm (red arrows) is implemented by writing to/reading from the sharepoint configuration and central administration content database. Communication between farms happens through the web services (dark blue arrows) and the http post request to do the file uploaded (light blue arrow):

When a content deployment job is started through the central admin UI a new one time timer job is created and executed (1). Alternatively the preconfigured timer job for a scheduled deployment job can be started by the timer service manually.

The timer job exports the configured content into a temp directory and compresses it into one or multiple cab files.

When the export is finished the timerjob calls the GetRemoteAdminServerUrl operation of the ContentDeploymentRemoteImport Web service to get the information about the import server to use (2).

The next step is that the timer job calls the CreateJob operation of the ContentDeploymentRemoteImport Web service to create the shadow content deployment job and the associated one time import timer job on the target farm. This job will remain in a stopped state (3/4).

Afterwards the timer job on the source farm uploads the exported cab files to the temp directory on the import server in the target farm using http upload to the DeploymentUpload.aspx (5).

After the upload is completed the timer job on the source farm calls the RunJob operation of the ContentDeploymentRemoteImport Web service to start the import timer job on the target farm (3/4).

While the import is running the import timer job will update the status of the deployment in the associated content deployment job list item.

Every 10 seconds (configurable though object model by setting the RemotePollingInterval setting – see part 2) the timer job on the source farm will call the GetJobStatus operation of the ContentDeploymentRemoteImport Web service to retrieve the status of the import operation (3/6). The timer job on the source farm will update the status in the content deployment job list item with the status received from the target farm. If the update hasn’t changed for 600 seconds (configurable though object model by setting the RemoteTimeout setting – see part 2) the timer job on the source farm assumes that something went wrong and will report a timeout.

When the timer job on the source server receives the information that the import operation has completed (either succeeded or failed) it will delete the shadow deployment job and the associated timer job through a call to the DeleteJob operations of the ContentDeploymentRemoteImport Web service (3/4)

The central administration retrieves the status of the timer job by looking at the content deployment job and job report list item which are updated by the timer service on the source farm (7). Every refresh of the page only reads the status in the local content deployment job list item. No call to the target farm to get a status update is done when refreshing this page.

9 Comments


  1. I have problem in content deployment when source server is https and destination is http, the job stuck at ‘preparing’ status, what is the solution?

    Thanks

    Reply

  2. Hi Jose,

    "Preparing" is before exporting starts. At this stage the protocol being used does not matter. If the job is stuck in "Preparing" then the timerjob does not start.

    Usually that means that the server configured as export server does not host the Central Admin website.

    Please double check.

    Cheers,

    Stefan

    Reply

  3. Hi Stefan,

    The content deployment job seems to fail when I have a document that is 100MB in size or bigger. I can upload this same file to the Source Central Administration site and as well as my target site without any problem. Do you know if the 100MB is a hard limit in SharePoint?

    Thanks

    Reply

  4. Hi Stefan,

    I am trying to deploy a sub site from site collection1 to sitecollection2 (sitecollection2 already has many subsites in it and same is the case with site collection1). The job is continuoulsy failing saying "remote upload web request failed."

    I also tried exporting a blank site from 1 to 2. Also tried creating a blank site at target site collection and then tried moving the existing blank site through job from 1 to 2.

    Also tried creating a blank site at target site collection and then tried moving the existing non-blank site through job from 1 to 2.  But always unsuccessful.

    I have tried through STSADM and Migration API and both work fine.

    have checked the following:

    1. No load balancing on server,

    2. Have made sure that application pool account and timer service account is same. The same account is part of power group, admin group, IIS_WPG group etc.  It also has db_Owner rights on relevant databases.

    Have tried using Test Job feature of a particular job and it Succeeds.

    Really need some help on this.

    Thanks,

    Pulkit Ojha

    Reply

  5. Hi Pulkit,

    the error means that the remote upload using the DeploymentUpload.aspx failed.

    It does not have to do with the way how you created the target site collection.

    Please check the IIS log of the central admin site of the import server and check for the http status code of the POST request going to the DeploymentUpload.aspx as this might give you more info about what is going wrong.

    Also check the ULS log of the target server for errors and exceptions.

    If this does not help you might want to take a network monitor trace to check the tcp communciation while performing the upload.

    Cheers,

    Stefan

    Reply

  6. Hi Stefan,

    I have had an intermittent content deployment failure problem in SP 2007 for a long time, in with about 7% of imports fail part way through importing with a stack trace containing "PatchUrl".  Rerunning the failed deployment nearly always succeeds. The ULS has not provided anything particularly useful for this error. IIS DeploymentUpload.aspx shows a 401.1 response immediately followed by a 200.0 response when it retries with the destination farm's service account, like this:

    2011-02-21 17:43:55 W3SVC1857002913 10.10.83.60 POST /_vti_adm/ContentDeploymentRemoteImport.asmx – 8989 – 10.10.34.55 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.3615) 401 1 0

    2011-02-21 17:43:57 W3SVC1857002913 10.10.83.60 POST /_vti_adm/ContentDeploymentRemoteImport.asmx – 8989 DMZsvc_SP_GFO 10.10.34.55 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.3615) 200 0 0

    2011-02-21 17:43:57 W3SVC1857002913 10.10.83.60 POST /_admin/Content+Deployment/DeploymentUpload.aspx filename=%22ExportedFiles.cab%22&remoteJobId=%22725020a4-ce37-4b17-96a2-9ab683a348ba%22 8989 – 10.10.34.55 – 401 1 0

    2011-02-21 17:43:59 W3SVC1857002913 10.10.83.60 POST /_admin/Content+Deployment/DeploymentUpload.aspx filename=%22ExportedFiles.cab%22&remoteJobId=%22725020a4-ce37-4b17-96a2-9ab683a348ba%22 8989 DMZsvc_SP_GFO 10.10.34.55 – 200 0 0

    Thoughts? We are on SP 2007 SP2 with the IU and April 2009 CU, and the problem happens on both WS 2008 and WS 2003 target farms.

    Reply

  7. Hi Erik,

    such an issue has been fixed in June 2010 CU.

    Cheers,

    Stefan

    Reply

  8. Hi Stefan,

    I have one problem. Our content deployment job exports 45 objects and imported 45 objects but it has completed with error.

    Accessed Denied error.  Please help me to know that what could be the root cause and how to resolve it.

    Sivanesan.t@gmail.com

    Thanks

    Reply

  9. Hi Siva,

    the information is not sufficient to isolate a Problem. Please open a case with Microsoft Support for further Troubleshooting of the issue.

    Cheers,

    Stefan

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.