Comments (14)

  1. Anonymous says:

    Hi Brad,

    the incremental push is done as a running service. Take a look into Windows "Services.msc" You should see a Service called "Replication Engine". What i've seen is, the incremental replication runs in every 5 secounds (i found a thread.sleep() statement using Reflector/ILSpy) that's the default schedule plan for incremental replication.

    Each schedule runs the replication with:

    1st: collect the chanageLog from MySite-Webapplication.

    2nd:. try to push all changes from changeLog to the target UPA.

    In case 14 days of change log is not large enough you have the changethe size of the change log under Central administration / General setting of the mysite webapplication.

    regards

    patrick

  2. Anonymous says:

    Hi RichM,

    based on logs i would say:

    [401.2 -> here you should see something in IIS http kernel log, or in SharePoint ULS log]

    POST /_vti_bin/userprofileservice.asmx – 80 – 10.7.2.223 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 401 2 5 258 752 0

    -> is 10.7.2.223 your server IP or client IP from UPRE?

    [that's ok, the first call is annonymous, 401.1]

    POST /_vti_bin/userprofileservice.asmx – 80 – 10.7.2.223 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 401 1 2148074254 539 424 0

    [that's the 2nd request with credential, and 200 is successful, -> no problem, UPRE run successful]

    POST /_vti_bin/userprofileservice.asmx – 80 <domain><username> <IP Address> HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 200 0 0 49618 1454 31

    at the end, you should focus on:

    [401.2 -> here you should see something in IIS http kernel log, or in SharePoint ULS log]

    POST /_vti_bin/userprofileservice.asmx – 80 – 10.7.2.223 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 401 2 5 258 752 0

    via: fiddler, IIS logs, IIS httpsys logs and ULS logs from sharepoint.

    in case you have a errror in sharepoint uls log then the callstack is very good to get more idear.

    regards

    Patrick

  3. Anonymous says:

    Hi RichM,

    using IIS Status code -> support.microsoft.com/…/en-us

    in your IIS log i can see:

    Line 1: 401 2 => 401.2 – Logon failed due to server configuration.

    Line 2: 401 1 => 401.1 – Logon failed.

    In this case i would check:

    1. authentication of the webapplication: MySite Webapplication should use Windows Authentication for UPRE, no forms auth., i would play with "classic mode" as easy as possible, if you can not change the settings then i would extend the webapplication for UPRE on a new URL. There you can play with new authentication stuff.

    2. is the mysite url configure as trusted zone in your Internet setting?

    3. get more idears:  increase your IIS logging Level: enable all checkboxes in IIS W3C logs, repro the issue and take a look into the W3C logs again, (right now you have only default logging configured)

    4. is you account authorized on mysite webapplication and on the UserProfile Service Application?

    5. and from step 1: what type of authentication is configure: claims mode vs. classic mode, Windows Authentication, Forms, ….

    regards

    patrick

  4. Anonymous says:

    Hi Jonny,

    please check the notification setting on each user Profile on both farms.

  5. Anonymous says:

    Hello RichM,

    http 401 is more then Unauthorized. As next step i would like see the substatus code from IIS W3C logs. What UPRE does is: connect to the source Url and call UserProfile webservice (.asmx) then UPRE connects to the target URL and call the next webservice.

    Check you IIS W3c Logs from Mysite webapplication. there you will find the 401.x substatus code.

    You manage the Webservice call permission on central admin / service applications / select you User Profile Application / press Manage Permission

    Your logon account starts cmd or powershell console. this account calls the UserProfile Webservice. on your IIS logs you will see the account too.

    regards

    Patrick

  6. Good post Patrick. Wish I have found it earlier and could have save some long hours.

    http://www.agileconcepts.com/…/Post.aspx

    Aamir Qureshi

  7. Nico de Jong says:

    This is one of the best articles on this subject – I'm though experiencing some diffuclty synchronising some custom properties – here is the thread in the sp2010 forum… maybe somebody comes a cross a solution to this…

    social.msdn.microsoft.com/…/8481c647-fc1c-4c87-85d6-f33c0c2f14ba

  8. brad says:

    Hi Patrick, thanks for this post, with incremental replication i read on technet that it uses a change log, and you must do the incremental push within 14 days, to pick up changes, do you know if this limit is fixed, or can be extended etc?

    i.e. you may hit a scenario whereby your incremental stops working and only notice this 3 weeks later by which time you would have lost a weeks worth of changes?

    thanks

    Brad

  9. brad says:

    Hi Patrick,

    Thanks for the reply, you seem to be one of the very few excellent resources i have found on this topic!

    I still have a couple of questions though following on from example in this article.

    Q1. If you have a global company that say has one company domain name let us call it "Widgets", but say has 3 mysite farms in 3 regions europe, america and china, each with the following 3 urls.

    http://Europe.demo.com

    http://America.demo.com

    http://China.demo.com

    We also have set up 3 audiences so each region has its trusted set of mysite users, lets say 3000 in each region so 9000 employees in total.

    Lets assume these farms have all been running for a while, and each farm contains a full set of profiles, for local people / profile search (so 9000 user profiles in each region, so someone in Europe can find contact details for someone in America).

    However you notice an issue… incremental sync has stopped working, users are continuing to update profiles and basically the farms are getting out of sync. Its got messy. So basically a full resync is necessary, you want to send selected profile fields from Europe to China and America and then repeat the process from the other 2 regions. You Basically get to a known state before starting incremental replication off again.

    So what I would like to do is do a  full push from one region just based on trusted mysite location. So Europe will update the user profile service application in china and america but only for the 3000 europe records, it will leave the other regional profiles (based on trusted mysite location) untouched.

    So how is this possible to do a full replication based on trusted mysite locations without causing a data overwrite in one of the farms? Whilst there is a parameter in the powershell -ActivityDirectoryDomains, I assume in my example above this would be  "widgets", but this wont help us as all 3 regions are using the same domain. They have different urls though for mysites, and I believe if you did this

    -source http://Europe.demo.com -destination http://America.demo.com

    it would still take all 9000 profiles from europe and overwrite all 9000 in America, not just the 3000 trused in Europe?

    Q2. On incremental changes if you have 3 farms, how does the change / audit log work to keep all 3 farms in sync without causing some form of cyclic infinite replication? From what I have read when you use incremental replication  you need to base it on trusted mysite locations, so if a user makes a change in their trusted mysite farm (europe), this is flagged as a change and lets say your scripts then send this change onto the america farm, however america cannot send this change onto china, as america can only send changes on based on its 3000 trusted mysite profiles? So China in theory then, would have to be updated from Europe again, but why would the europe system, send this change on again, because the change log already been updated to say this item has been processed when it was first sent to America? or does the audit log keep a track of each change and where it was pushed to?

    Q3.As you pointed out above in your reply to me, you can change the change audit log retention time. Our production farm is set to 60 days (not sure if thats the default),  but if via incremental replication it is processing it every 5 secs, why would you keep so much? i.e 60 days worth, is it just incase an issue occurs? Even if you did have an issue how would you know or the system know, where to start back from? Is this audit / change used by other sharepoint applications / jobs or just for UPRE? There is also an option to set it to Never, in which case I guess you would only ever set it to this, if you were not using UPRE?

    Q4. for incremental replication there is a parameter called "Feed Properties", which is Optional! and based on trusted mysite location, why would this be optional though, as without using trusted mysites the incremental would get in some cyclic loop?

    Sorry for the long post!! I owe you a drink if you have got this far! but just trying to set the context / scenarios and understand this thing! so we dont cause data corruption! Any help appreciated.

    thanks

    Brad

  10. jonny says:

    Hi, great article. I disabled the email notifications and ran the full import but the emails were still sent!

  11. RichM says:

    Hi Patrick,

    Great article, especially as there's not a lot of stuff out there about the UPRE.

    My issue is that I can get the UPRE to perform a Full Replication but in the Full_Err_Push log every user account in the source User Profile Service errors with:

    System.Net.WebException: The request failed with HTTP status 401: Unauthorized.

    If I then try to run a Full Replication again, the actual PowerShell window errors with:

    Start-SPProfileServiceFullReplication : The request failed with HTTP status 401

    : Unauthorized.

    I've checked all the obvious stuff like permissions for the account running the service and the loopback check but try as I might I cannot get the UPRE to subsequently run.

  12. RichM says:

    Hi Patrick,

    Thanks again for the speedy reply:

    1. MySite IIS Logs – I have generated the error again and the following two lines get logged with 401 codes:

    2012-08-23 14:28:28 ::1 GET / – 999 – ::1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/4.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729) 401 2 5 0

    2012-08-23 14:28:28 ::1 GET / – 999 – ::1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/4.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729) 401 1 2148074254 0

    Many thanks again

  13. RichM says:

    Hi Patrick,

    Many thanks again for your help.

    At the moment I can answer 1, 2, 4 & 5

    1. Both the source and destination MySite Web Applications are using Windows Authentication

    2. They weren't, but they are now. It hasn't made a difference to the situation.

    4. Yes – in both places. Just for extra info, both Farms are in the same domain and I'm using the same account on both farms. The account is a Farm Admin in both farms as well as having Full Control of the MySite Web Apps and the User Profile Service

    5. see 1

    For some reason today I'm not getting any IIS logs produced and at the moment I cannot figure out why. I've increased the log level on the soure MySite Web Application (ticked all the boxes in the select fields section) but alas no log file – I'll keep trying and as soon as I have one I'll post the details……………

    Still for the life of me cannot figure why it ran once (even though it didn't replicate any profiles) but will not run again when nothing has changed! Grrrrrr!

  14. RichM says:

    Hi Patrick,

    I've got my IIS logs going again but even though I have selected every field I'm getting nothing new in the logs.

    I get the two lines with the 401 error as before and one line with a 200 code (which I believe is success:

    POST /_vti_bin/userprofileservice.asmx – 80 – 10.7.2.223 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 401 2 5 258 752 0

    POST /_vti_bin/userprofileservice.asmx – 80 – 10.7.2.223 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 401 1 2148074254 539 424 0

    POST /_vti_bin/userprofileservice.asmx – 80 <domain><username> <IP Address> HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+MS+Web+Services+Client+Protocol+2.0.50727.5448) – – mysite 200 0 0 49618 1454 31

Skip to main content