Troubleshooting ASR Initial Replication issues

  • Article

 'My Initial replication is failing' or 'My Initial replication is stuck at 0%'

Most of the initial replication failures that we encounter at support are due to connectivity issue between Source Server-to-Process Server or Process Server-to-Azure as you can see from this sample cases:

  • My replication was stuck at 0% for a couple days. Initial Replication (IR) was going slowly due to low bandwidth and warnings were thrown.
  • My replication was stuck at 0% due to Stale entry of host
  • My replication was stuck at 0% due to Network issue
  • My replication was stuck at 0%, restarting CBengine service resolved the issue
  • My replication was stuck at 0%, because my Processor storage was 80% filled causing IR to stop
  • My replication was stuck because WMI is not working on server . Restarted the host server
  • My replication was stuck at 0% and 55% due to low bandwidth; however, it got resolved itself
  • My replication was stuck at 0%, because inbound port 9443 on our Process Server was blocked
  • My enable replication showed status at 0% synchronized, fixing the Proxy configuration resolved the issue

For most cases you can self troubleshoot these issues by following the steps listed below.  If it doesn't resolve your issue, then post your query to ASR forum. We have an active community and one our engineers will be able to assist you.

Step 1: Check connectivity between Source Server-to-Process Server

  • From Source Server machine command line use Telnet to ping the Process Server with https port (default 9443) as shown below to see if there are any network connectivity issues or firewall port blocking issues.

        telnet <PS IP address> <port>

    Note: Use Telnet, don't use PING to test connectivity.  If Telnet is not installed, follow the steps list here.

    If unable to connect, allow inbound port 9443 on the Source machine and check if the problem still exits.

  • Check the status of following Service.
         InMage Scout VX Agent - Sentinel/OutpostStart if it is not running and check if the problem still exists.

Step 2: Check connectivity between Process Server-to-Azure

  • From Process Server machine, open the Task Manager (press Ctrl-Shift-Esc ).  Go to the Performance tab and click 'Open Resource Monitor' link. From Resource Manager, go to Network tab.  Check if cbengine.exe in 'Processes with Network Activity' is actively sending large volume (in Mbs) of data.
    cbengine
    If not follow the steps listed below.

  • Select and check cbengine.exe to view the 'TCP Connections' to see if there is connectivity from Process server to Azure Storage blob URL.
    resourcemonitor
    If not follow the steps listed below.

  • From Control Panel > Services, check if the following services are up and running:
         cxprocessserver
    InMage Scout VX Agent - Sentinel/Outpost
         Microsoft Azure Recovery Services Agent
         Microsoft Azure Site Recovery Service
         tmansvc
    (Re)Start any service which is not running and check if the problem still exists.Note: for some customers, Re-Starting Microsoft Azure Recovery Services Agent service is said to resolve their issue.

  • Open the latest CBEngineCurr.errlog from %programfiles%\Microsoft Azure Recovery Services Agent\Temp and search for :443  and connection attempt failed.

    logdetails1

    If there are issues, then from Process Server command line use  telnet to ping your Azure Public IP address (masked in above image) found in the CBEngineCurr.currLog using port 443.

           telnet <your Azure Public IP address as seen in CBEngineCurr.errlog>  443

    If you are unable to connect, then check if the access issue is due to firewall or Proxy as described in next step.

  • Process server must have internet access.  If there are issues, check firewall rules or Proxy settings to enable access.

    • If you are using an IP address-based firewall rules on the server, then download the complete list of Microsoft Azure Datacenter IP Ranges from here and add them to your firewall configuration to ensure they allow communication to Azure (and the HTTPS (443) port).  Allow IP address ranges for the Azure region of your subscription, and for West US (used for Access Control and Identity Management).

    • If you are using an URL based firewall rules on the server, ensure the following URLs are added to firewall configuration.  *.accesscontrol.windows.net: Used for access control and identity management
        \*.backup.windowsazure.com: Used for replication data transfer and orchestration
        \*.blob.core.windows.net: Used for access to the storage account that stores replicated data
        \*.hypervrecoverymanager.windowsazure.com: Used for replication management operations and orchestration
        time.nist.gov and time.windows.com: Used to check time synchronization between system and global timeURLs for Azure Government Cloud:

        • .ugv.hypervrecoverymanager.windowsazure.us
        • .ugv.backup.windowsazure.us
        • .ugi.hypervrecoverymanager.windowsazure.us
        • .ugi.backup.windowsazure.us

    • If you are using a Proxy Server, ensure the steps outlines in 'Configure Outgoing Network' listed in this comprehensive article are followed (here is a section from the article for configuring this on Windows Server).

            This will setup proxy server configuration for Local System Account.

        1. Download PsExec

        2. Run following command from elevated prompt,
            psexec -i -s "c:\Program Files\Internet Explorer\iexplore.exe"

          It will open internet explorer window.

        3. Go to Tools -> Internet Options -> Connections -> LAN settings.

        4. Verify proxy settings for System account. Set Proxy IP and port.

        5. Close Internet Explorer.

      This will set up a machine-wide proxy configuration, and will be used for any outgoing HTTP/HTTPS traffic.

      Also ensure, the proxy server name is resolving by the DNS server.

    • In addition, the proxy server should allow access to the following URLs: *.accesscontrol.windows.net: Used for access control and identity management
      \*.backup.windowsazure.com: Used for replication data transfer and orchestration
      \*.blob.core.windows.net: Used for access to the storage account that stores replicated data
      \*.hypervrecoverymanager.windowsazure.com: Used for replication management operations and orchestration
      time.nist.gov and time.windows.com: Used to check time synchronization between system and global time

  • Finally, check if Throttle bandwidth is constrained.  Increase the bandwidth (ex. greater than 20Mbs) and check if the problem still exists.If your issue is not yet resolved, post your query to ASR forum. We have an active community and one our engineers will be able to assist you.