Troubleshooting Tip for Debugging Crawl Issues in SharePoint 2010


I recently came across a very nice troubleshooting methodology when I was trying to debug some authentication issues that were occurring during a SharePoint 2010 crawl.  I was getting some errors and also having difficulty getting the information I needed out of the crawl log to some other issues that were occurring.  Strangely enough, enter Fiddler to the rescue (www.fiddler2.com).

 

I’m sure most everyone is familiar with Fiddler so I won’t bother explaining it here.  The trick though was to get it to capture what was happening during the crawl.  I found a very slick way to do this is to configure Fiddler as a reverse proxy for the crawl account.  The instructions for configuring Fiddler as a reverse proxy can be found here:  http://www.fiddler2.com/Fiddler/help/reverseproxy.asp.  The way I used it was as follows:

 

  1. Log into my crawl server as the crawl account.
  2. Configure Fiddler to be a reverse proxy as described above.
  3. Start Fiddler
  4. Start a new crawl.

 

I had isolated my trouble sites into a separate content source.  So once I followed these steps I was able to see each request from the crawler to that content source, how it was authenticating, and what was happening.  Overall it proved very useful in understanding much more clearly what was going on during the crawl of those sites.

Comments (7)

  1. alexandrad9x says:

    Tao http://dichvuketoanlongbien.com/
    Rủa
    http://dichvuketoanlongbien.com/a2-96-dich-vu-ke-toan-tron-goi.html
    Thằng http://dichvuketoanlongbien.com/a2-98-dich-vu-ke-toan-thue.html
    Cờ
    http://dichvuketoanlongbien.com/a2-103-dich-vu-bao-cao-tai-chinh.html
    http://dichvuketoanlongbien.com/a2-97-dich-vu-quyet-toan-thue.html
    Nào
    http://dichvuketoanlongbien.com/a2-114-dich-vu-ke-toan-tai-29-quan-huyen.html
    Soi
    http://dichvuketoanlongbien.com/i780-dich-vu-ke-toan-thue-tron-goi-tai-bac-ninh.html
    Tài
    http://dichvuketoanlongbien.com/i779-dich-vu-ke-toan-thue-tron-goi-tai-bac-giang.html
    Khoản
    http://dichvuketoanlongbien.com/i778-dich-vu-ke-toan-thue-tron-goi-tai-phu-tho.html

    http://dichvuketoanlongbien.com/i781-dich-vu-ke-toan-thue-tron-goi-tai-hung-yen.html
    Link
    http://dichvuketoanlongbien.com/i782-dich-vu-ke-toan-thue-tron-goi-tai-vinh-phuc.html
    Của
    http://dichvuketoanlongbien.com/i783-dich-vu-ke-toan-thue-tron-goi-tai-hai-phong.html
    Tao. http://www.trungtamketoan.com.vn/
    Chúng
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-ha-noi.html
    Mày
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-tp-hcm.html
    Đủ
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-quang-ninh.html
    Trình
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-hai-duong.html
    Thì
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-bac-giang.html
    Tự
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-bac-ninh.html
    Đi
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-hai-phong.html

    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-nam-dinh.html
    Làm.
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-thai-binh.html
    Việc
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-thanh-hoa.html

    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-vinh-phuc.html
    Phải
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-hung-yen.html
    Rẻ
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-phu-tho.html
    Rách
    http://www.trungtamketoan.com.vn/p/trung-tam-dao-tao-ke-toan-tai-binh-duong.html
    Như http://www.tosvn.com
    Thế. http://iketoan247.blogspot.com
    Loại http://tailieuveketoan.blogspot.com
    Chó http://mauhinhnendep.blogspot.com
    Má. http://www.tosvn.com/search/label/Hack%20CF
    Tao http://www.tosvn.com/search/label/Hack%20AvatarStar
    Rủa http://www.tosvn.com/search/label/Hack%20Warcraft-Dota2
    Những http://hocketoan360.com/category/tai-lieu-ke-toan/
    Thằng http://iketoan247.blogspot.com/search/label/thong-tin-kinh-te
    Soi http://iketoan247.blogspot.com/search/label/tin-bai-ve-thue
    Tao http://hoclamketoan.edu.vn/
    Sẽ http://hoclamketoan.edu.vn/category/khoa-hoc-ke-toan
    Tan http://hoclamketoan.edu.vn/category/dich-vu-ke-toan
    Cửa http://hoclamketoan.edu.vn/category/hoc-lam-ke-toan
    Nát http://hoclamketoan.edu.vn/category/tai-lieu-ke-toan
    Nhà http://hocketoan360.com/
    Haha http://hocketoan360.com/category/khoa-hoc-ke-toan/
    http://hocketoan360.com/category/dich-vu-ke-toan/

  2. chris.mullendore@microsoft.com says:

    Very few people realize how useful a proxy server can be. For example, you can also ask it to get involved in SOAP conversations too if necessary. WAY easier to read than a network trace.

  3. yvan says:

    Hello Steve,

    I found your tip really useful and I'm trying to use it but I can't understand how you manage to make it working if you don't change the port of the website you want to crawl in the content source?

  4. Kalashnikov says:

    Hey,

    This article might help to troubleshoot your issue..

    kalashnikovtechnoblogs.blogspot.in/…/troubleshooting-crawl-issues-for-ssl.html

    Thanks,

    Kalashnikov

    kalashnikovtechnoblogs.blogspot.in

    if it helps, Please dont forget to mark it as an answer. Thank you.

  5. DSF says:

    http://www.shopbestgoods.com/
    http://www.nike-jordanshoes.com/
    http://www.beatsbydreoutlet.net/
    http://www.michaelkorsus.com/
    http://www.polo-tshirts.com/
    http://www.northsclearance.com/
    http://www.ralph-laurensale.com/
    http://www.gucci-shoesuk2014.com/
    http://www.michael-korsusa.com/
    http://www.polo-outlets.com/
    http://www.ralphslauren.co.uk/
    http://www.marcjacobsonsale.com/
    http://www.mcmworldwides.com/
    http://www.salongchamppairs.com/
    http://www.canada-gooser.com/
    http://www.burberryoutlet2014.com/
    http://www.michaelkors.so/
    http://www.hermes-outletonline.com/
    http://www.oakley-sunglassoutlet.com/
    http://www.north-faceoutlets.net/
    http://www.moncler-clearance.com/
    http://www.woolrich-clearance.com/
    http://www.barbour-jacketsoutlet.com/
    http://www.moncler-jacketsoutletonline.com/
    http://www.monsterbeatsbydres.net/
    http://www.louis-vuittonblackfriday.com/
    http://www.lv-guccishoesfactory.com/
    http://www.mcmoutlet-jp.com/
    http://www.cheapdiscountoutlet.com/
    http://coachoutlet.iwopop.com/
    http://www.coachsfactoryoutlet.com/
    http://www.coach-blackfriday2014.com/
    http://www.coach-storeoutletonline.com/
    http://www.coach-factorysoutletonline.com/
    http://www.coachccoachoutlet.com/
    http://www.coach-factories.net/
    http://www.coach-pursesoutletonline.com/
    http://www.llouisvuitton-factory.net/
    http://www.coach-outletsusa.com/
    http://www.mksfactoryoutlet.com/
    http://www.zxcoachoutlet.com/
    http://www.mischristmas.com/
    http://www.misblackfriday.com/
    http://www.bestcustomsonline.com/
    http://www.newoutletonlinemall.com/
    http://www.clickmichaelkors.com/
    http://www.cmichaelkorsoutlet.com/
    http://www.ralphlaurenepolo.com/
    http://michaelkorsoutlet.mischristmas.com/
    http://mcmbackpack.mischristmas.com/
    http://monsterbeats.mischristmas.com/
    http://northfaceoutlet.mischristmas.com/
    http://mk.misblackfriday.com/
    http://coachoutlet.misblackfriday.com/
    http://coachfactory.misblackfriday.com/
    http://uggaustralia.misblackfriday.com/
    http://coachpurses.misblackfriday.com/
    http://coachusa.misblackfriday.com/
    http://coach.misblackfriday.com/
    http://michaelkorss.misblackfriday.com/
    http://michaelkors.misblackfriday.com/
    http://airmax.misblackfriday.com/
    http://michael-kors.misblackfriday.com/

    http://t.co/1PJuejI1ys
    http://t.co/FYm2MxWwLM
    https://twitter.com/CoachOutlet2014
    https://www.facebook.com/pages/Coach-Factory-Outlet-Online-Store-Michael-Kors-Outlet-Online-Sale-75-Off/712060898859091
    https://www.facebook.com/pages/Ralph-Lauren-Polo-Outlet-Online-Sale/1404100279810690

  6. m88 says:

    MU has welcomed the news could not be happier M88:
    http://www.m88no.com/Main/Home.aspx?affiliateId=99156
    captain Wayne Rooney injury has recovered and will attend the match against Manchester City on Sunday. Etihad pitch certainly be concerned by the arrival of England striker here always brings a nightmare for the home side.

    FOOTBALL QUICK UPDATE BEST 24H, 15 MINUTES AFTER THE GAME HAS ENDED MAY M88 VIDEO CLIP FOOTBALL AND SPORTS

    Rooney: The King of derby
    Rooney has scored 11 goals in 22 games for Manchester derby and that’s why coach Louis van Gaal are anxiously awaiting return of Reds skipper.

    If the fulcrum of England’s champions, the fulcrum Etihad mecca of Manchester in Manchester City’s encounter was "Shrek guy". Three Lions striker has 7 goals in the game, the team and the city M88.