Testing RPC ports with PowerShell (and yes, it’s as much fun as it sounds)


What do you do when you have to troubleshoot the dreaded "RPC Unavailable" error 1722, which rears its ugly head anywhere from Active Directory replication to Configuration Manager Distribution Point installations, and many other places in between?

Example of the Error 1722 RPC Server Unavailable in the wild

 

To help prevent this from becoming a blame pointing cage match between the System Admins and the Network Folks (we all know the Sys Admins would win anyway), I'm writing this post to help put some structure around troubleshooting this type of issue so that you have more facts to take to the conversation. "I think it's a firewall issue" doesn't get you very far in my experience.

RPC communication is one of the tougher firewall problems since most firewall folks want to know exactly which ports you need open.  With RPC, they get a range of ports from 49152 to 65535.  There are usually predefined rules on firewalls, WAN accelerators, and the various devices that traffic hops through to get to its destination.  They don't always work as planned.

To give you the simplest example I can think of, RPC sort of works like the concierge desk at a hotel.  You walk up and ask the person at the desk for the information about services at the hotel, like the gym or the swimming pool.  In our scenario, that person at the desk is RPC Endpoint Mapper on port 135 and they direct you to the services that are listening on the ephemeral ports.

I'm just barely breaking the surface on RPC in this post.  If you want to get the full picture, take a look at what, in my opinion, is the best explanation about how RPC works in gory detail, written by Ned Pyle (and his dogs) at the link below:

https://blogs.technet.microsoft.com/askds/2012/01/24/rpc-over-itpro/

I see a lot of administrators attempting to diagnose this network connectivity by looking up the RPC ports (135 & 49152-65535) and then attempting to connect to random ports in the ephemeral range, hopefully this post will help with isolating the ports that are truly listening on the server.

Getting the ports

The way I normally troubleshoot this type of network connectivity is with the SysInternals PortQry.exe utility, which can be downloaded from the Microsoft website.

To begin, run the following command to query the RPC Port Mapper on the remote machine, this will return the ports in the ephemeral range that the machine is actively listening on for RPC services:

Portqry.exe -n 169.254.0.10 -e 135

(PARTIAL OUTPUT BELOW)

Querying target system called:
 169.254.0.10
Attempting to resolve IP address to a name...
IP address resolved to DC1.contoso.com
querying...
TCP port 135 (epmap service): LISTENING
Using ephemeral source port
Querying Endpoint Mapper Database...
Server's response:
UUID: d95afe70-a6d5-4259-822e-2c84da1ddb0d
ncacn_ip_tcp:169.254.0.10[49664]
UUID: 50abc2a4-574d-40b3-9d66-ee4fd5fba076
ncacn_ip_tcp:169.254.0.10[64555]
UUID: 897e2e5f-93f3-4376-9c9c-fd2277495c27 Frs2 Service
ncacn_ip_tcp:169.254.0.10[64528]
UUID: 367abb81-9844-35f1-ad32-98f038001003
ncacn_ip_tcp:169.254.0.10[64502]
UUID: c9ac6db5-82b7-4e55-ae8a-e464ed7b4277 Impl friendly name
ncacn_ip_tcp:169.254.0.10[49668]
UUID: 12345778-1234-abcd-ef00-0123456789ac
ncacn_ip_tcp:192.168.0.242[49668]

The output from this command will have an almost overwhelming amount of output.  Within this mountain of data will be just a handful of high-numbered ephemeral ports on which the server is listening.  You are looking for any lines that have "ip_tcp" in them and the ports are in brackets at the end of the line (highlighted in blue above).  This is where we will get a focused list of listening ports from the RPC server to query and validate connectivity.

In the abbreviated example above, ports 49664 , 64555, 64502,and 49668 are listening.  Note that 49668 is listed twice.  There will be duplicates that you will have to filter out.  Once you have the full (and de-duplicated) list put together, you can then feed that list of ports back into PORTQRY.EXE to validate that they are reachable over the network.

About now, you are probably saying to yourself that's a  lot of work!  Never fear, I didn't do this too many times before I decided to automate it in PowerShell.

RPCCheck.ps1 to the rescue!

#  This Sample Code is provided for the purpose of illustration only and is not intended to be used in a production environment.
#  THIS SAMPLE CODE AND ANY RELATED INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR   
#  FITNESS FOR A PARTICULAR PURPOSE.
#
#
#  Script queries port 135 to get the listening ephemeral ports from the remote server
#  and verifies that they are reachable.  
#
#
#  Usage:  RPCCheck -Server YourServerNameHere
#
#
#  Note:  The script relies on portqry.exe (from Sysinternals) to get port 135 output.
#  The path to portqry.exe will need to be modified to reflect your location
#  


Param(
  [string]$Server
)

#  WORKFLOW QUERIES THE PASSED ARRAY OF PORTS TO DETERMINE STATUS

workflow Check-Port {

  param ([string[]]$RPCServer,[array]$arrRPCPorts)

  $comp = hostname

 

  ForEach -parallel ($RPCPort in $arrRPCPorts)
  {

      $bolResult = InlineScript{Test-NetConnection -ComputerName $Using:RPCServer -port $Using:RPCPort _
-InformationLevel Quiet}

 If ($bolResult)
 {
      Write-Output "$RPCPort on $RPCServer is reachable"
 }
 Else
 {
     Write-Output "$RPCPort on $RPCServer is unreachable"
 }
}
}

#  INITIAL RPC PORT

$strRPCPort = "135"

#  MODIFY PATH TO THE PORTQRY BINARY IF NECESSARY
$strPortQryPath = "C:\Sysinternals"

#  TEST THE PATH TO SEE IF THE BINARY EXISTS
If (Test-Path "$strPortQryPath\PortQry.exe")
{
  $strPortQryCmd = "$strPortQryPath\PortQry.exe -e $strRPCPort -n $Server"
}
Else
{
  Write-Output "Could not locate Portqry.exe at the path $strPortQryPath"
  Exit
}

#  CREATE AN EMPTY ARRAY TO HOLD THE PORTS RETURNED FROM THE RPC PORTMAPPER
$arrPorts = @()

#  RUN THE PORTQRY COMMAND TO GET THE EPHEMERAL PORTS

$arrQuryResult = Invoke-Expression $strPortQryCmd

# CREATE AN ARRAY OF THE PORTS
ForEach ($strResult in $arrQuryResult)
{
  If ($strResult.Contains("ip_tcp"))
  {
  $arrSplt = $strResult.Split("[")
  $strPort = $arrSplt[1]
  $strPort = $strPort.Replace("]","")
  $arrPorts += $strPort
  }
}

#  DE-DUPLICATE THE PORTS
$arrPorts = $arrPorts | Sort-Object |Select-Object -Unique

#  EXECUTE THE WORKFLOW TO CHECK THE PORTS
Check-Port -RPCServer $Server -arrRPCPorts $arrPorts

The Results

Below is the output from the script in both a good and bad test:

Notes

Some things to note regarding the script:  It uses PortQry, so make sure you have that installed.  You'll have to tell the script where the PortQry.exe binary is located by modifying the path on this line of the script:

$strPortQryPath = "C:\Sysinternals"

Also, the script requires PowerShell v4 since it was written to use WorkFlow and Test-NetConnection, which requires PowerShell v3 & PowerShell v4 respectively.  

Finally

At the end of all of this, our family lion of a house cat (rescued from the Humane Society by the way), Roger, was very impressed. You can just see it on his face. Ned, in the off chance that you read this, I'm a dog person all the way, but the daughter wanted a cat.  If it helps, I asked for the "most dog-like" cat they had, and Roger doesn't disappoint.

Comments (0)

Skip to main content