SCCM client push installation may fail due to firewall problems

I was collaborating with a colleague of mine on a problem where SCCM client push installation was failing. They suspected network connectivity problems and collected simultaneous network traces from SCCM server and from a problem client machine and involved me in for further analysis.

 

When I check the SCCM server and client side traces, I saw that SCCM server was successfully accessing the client through TCP port 135

 

=> SCCM server side trace:

 

- TCP three way handshake between SCCM server and client:

 

5851 14:42:47 05.09.2012 34.0337296 10.0.9.149 CLIENTNAME.company.com TCP TCP: [Bad CheckSum]Flags=......S., SrcPort=51763, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=2250995253, Ack=0, Win=8192 ( Negotiating scale factor 0x8 ) = 8192 {TCP:861, IPv4:843}

5852 14:42:47 05.09.2012 34.0364843 CLIENTNAME.company.com 10.0.9.149 TCP TCP:Flags=...A..S., SrcPort=DCE endpoint resolution(135), DstPort=51763, PayloadLen=0, Seq=1315818582, Ack=2250995254, Win=65535 ( Negotiated scale factor 0x0 ) = 65535 {TCP:861, IPv4:843}

5853 14:42:47 05.09.2012 34.0365076 10.0.9.149 CLIENTNAME.company.com TCP TCP: [Bad CheckSum]Flags=...A...., SrcPort=51763, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=2250995254, Ack=1315818583, Win=258 (scale factor 0x8) = 66048 {TCP:861, IPv4:843}

 

- SCCM server binds to SCMActivator and activates WMI component:

 

5877 14:42:47 05.09.2012 34.0610846 10.0.9.149 CLIENTNAME.company.com MSRPC MSRPC:c/o Bind: IRemoteSCMActivator(DCOM) UUID{000001A0-0000-0000-C000-000000000046} Call=0x3 Assoc Grp=0xBB15 Xmit=0x16D0 Recv=0x16D0 {MSRPC:865, TCP:861, IPv4:843}

5880 14:42:47 05.09.2012 34.0642128 CLIENTNAME.company.com 10.0.9.149 TCP TCP:Flags=...A...., SrcPort=DCE endpoint resolution(135), DstPort=51763, PayloadLen=0, Seq=1315818583, Ack=2250996747, Win=65535 (scale factor 0x0) = 65535 {TCP:861, IPv4:843}

5882 14:42:47 05.09.2012 34.0748352 CLIENTNAME.company.com 10.0.9.149 MSRPC MSRPC:c/o Bind Ack: Call=0x3 Assoc Grp=0xBB15 Xmit=0x16D0 Recv=0x16D0 {MSRPC:865, TCP:861, IPv4:843}

5883 14:42:47 05.09.2012 34.0750212 10.0.9.149 CLIENTNAME.company.com MSRPC MSRPC:c/o Alter Cont: IRemoteSCMActivator(DCOM) UUID{000001A0-0000-0000-C000-000000000046} Call=0x3 {MSRPC:865, TCP:861, IPv4:843}

5884 14:42:47 05.09.2012 34.0785470 CLIENTNAME.company.com 10.0.9.149 MSRPC MSRPC:c/o Alter Cont Resp: Call=0x3 Assoc Grp=0xBB15 Xmit=0x16D0 Recv=0x16D0 {MSRPC:865, TCP:861, IPv4:843}

5885 14:42:47 05.09.2012 34.0786863 10.0.9.149 CLIENTNAME.company.com DCOM DCOM:RemoteCreateInstance Request, DCOM Version=5.7 Causality Id={FEEE1975-B61E-42EB-B500-939EA5EE4B2A} {MSRPC:865, TCP:861, IPv4:843}

  Frame: Number = 5885, Captured Frame Length = 923, MediaType = ETHERNET

+ Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[00-22-90-E3-B7-80],SourceAddress:[00-22-64-08-91-A6]

+ Ipv4: Src = 10.0.9.149, Dest = 10.102.0.230, Next Protocol = TCP, Packet ID = 639, Total IP Length = 909

+ Tcp: [Bad CheckSum]Flags=...AP..., SrcPort=51763, DstPort=DCE endpoint resolution(135), PayloadLen=869, Seq=2250996924 - 2250997793, Ack=1315818870, Win=257 (scale factor 0x8) = 65792

+ Msrpc: c/o Request: IRemoteSCMActivator(DCOM) {000001A0-0000-0000-C000-000000000046} Call=0x3 Opnum=0x4 Context=0x1 Hint=0x318

- DCOM: RemoteCreateInstance Request, DCOM Version=5.7 Causality Id={FEEE1975-B61E-42EB-B500-939EA5EE4B2A}

  + HeaderReq: DCOM Version=5.7 Causality Id={FEEE1975-B61E-42EB-B500-939EA5EE4B2A}

  + AggregationInterface: NULL

  - ActivationProperties: OBJREFCUSTOM - {000001A2-0000-0000-C000-000000000046}

   + MInterfacePointerPtr: Pointer To 0x00020000

   - Interface: OBJREFCUSTOM - {000001A2-0000-0000-C000-000000000046}

    + Size: 744 Elements

      InterfaceSize: 744 (0x2E8)

    - Interface: OBJREFCUSTOM - {000001A2-0000-0000-C000-000000000046}

       Signature: 1464812877 (0x574F454D)

       Flags: OBJREFCUSTOM - Represents a custom marshaled object reference

       MarshaledInterfaceIID: {000001A2-0000-0000-C000-000000000046}

     - Custom:

        ClassId: {00000338-0000-0000-C000-000000000046}

        ExtensionSize: 0 (0x0)

        ObjectReferenceSize: 704 (0x2C0)

      - ActivationProperties:

         TotalSize: 688 (0x2B0)

         Reserved: 0 (0x0)

       + CustomHeader:

       - Properties: 6 Property Structures

        + Special:

        - Instantiation:

         + Header:

           InstantiatedObjectClsId: {8BC3F05E-D86B-11D0-A075-00C04FB68820} => This is WMI

           ClassContext: 20 (0x14)

           ActivationFlags: 2 (0x2)

           FlagsSurrogate: 0 (0x0)

 

- Server responds with success and provides the endpoint information for WMI service:

 

5886 14:42:47 05.09.2012 34.0848992 CLIENTNAME.company.com 10.0.9.149 DCOM DCOM:RemoteCreateInstance Response, ORPCFLOCAL - Local call to this computer {MSRPC:865, TCP:861, IPv4:843}

        - ScmReply:

         + Header:

         + Ptr: Pointer To NULL

         + RemoteReplyPtr: Pointer To 0x00106E98

         - RemoteReply:

            ObjectExporterId: 13300677357152346811 (0xB8957F961925A2BB)

          + OxidBindingsPtr: Pointer To 0x00102FF0

            IRemUnknownInterfacePointerId: {0000B400-0580-0000-9A5E-C2357038B9DF}

            AuthenticationHint: 4 (0x4)

          + Version: DCOM Version=5.7

          - OxidBindings:

           + Size: 378 Elements

           - Bindings:

              WNumEntries: 378 (0x17A)

              WSecurityOffsets: 263 (0x107)

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\PIPE\\atsvc]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\PIPE\\wkssvc]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\pipe\\keysvc]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\PIPE\\srvsvc]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\pipe\\trkwks]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\PIPE\\W32TIME]

            - StringBindings:

               TowerId: 15 (0xF)

               NetworkAddress: \\\\CLIENTNAME[\\PIPE\\ROUTER]

            - StringBindings:

               TowerId: 7 (0x7)

               NetworkAddress: CLIENTNAME[1431]

            - StringBindings:

               TowerId: 7 (0x7)

               NetworkAddress: 10.102.0.230[1431]

              Terminator1: 0 (0x0)

            + SecurityBindings:

            + SecurityBindings:

            + SecurityBindings:

            + SecurityBindings:

            + SecurityBindings:

              Terminator2: 0 (0x0)

 

- Since WMI listens on TCP 1431, SCCM server tries to connect to that endpoint to access WMI subsystem:

 

...

8980 14:43:08 05.09.2012 55.1014127 10.0.9.149 CLIENTNAME.company.com TCP TCP: [Bad CheckSum]Flags=......S., SrcPort=51785, DstPort=1431, PayloadLen=0, Seq=1764982397, Ack=0, Win=8192 ( Negotiating scale factor 0x8 ) = 8192 {TCP:1203, IPv4:843}

9390 14:43:11 05.09.2012 58.1101896 10.0.9.149 CLIENTNAME.company.com TCP TCP:[SynReTransmit #8980] [Bad CheckSum]Flags=......S., SrcPort=51785, DstPort=1431, PayloadLen=0, Seq=1764982397, Ack=0, Win=8192 ( Negotiating scale factor 0x8 ) = 8192 {TCP:1203, IPv4:843}

11236 14:43:17 05.09.2012 64.1163158 10.0.9.149 CLIENTNAME.company.com TCP TCP:[SynReTransmit #8980] [Bad CheckSum]Flags=......S., SrcPort=51785, DstPort=1431, PayloadLen=0, Seq=1764982397, Ack=0, Win=8192 ( Negotiating scale factor 0x8 ) = 8192 {TCP:1203, IPv4:843}

 

- But this TCP session request fails because SCCM server doesn’t get a response to TCP SYN requests.

- When we check the client side network trace, we cannot see any of those TCP SYNs sent by the SCCM server.

 

This is most of the time a hardware router/firewall filtering problem. After our customer made the necessary configuration changes in the firewall, SCCM client push installation started working properly.

 

Since WMI is assigned a random TCP port from dynamic RPC port range at every startup, network/firewall administrators need to allow that range as well in addition to allowing TCP 135 activity towards the clients. One other alternative in this instance could be fixing the TCPIP port than WMI subsytem obtains at each startup. You can see the below article for more information on this:

 

https://support.microsoft.com/kb/897571 FIX: A DCOM static TCP endpoint is ignored when you configure the endpoint for WMI on a Windows Server 2003-based computer

 

Hope this helps

 

Thanks,

Murat