Hybrid Cloud Networking Help with NTttcp Version 5.28

Dr. Tom ShinderWith the recent release of Azure Infrastructure Services we anticipate that a lot of you interested in the Cloud and Datacenter Solutions HubMicrosoft cloud will be curious about the possibility of moving current on premises workloads to the Azure IaaS offering. There are a lot of good reasons to do this, such as potential cost savings and reduction of management overhead for the physical hardware. We anticipate the in the next few years that most enterprise IT organizations are going to be in hybrid mode, where on premises datacenters or private clouds will connect to resources in a public cloud.

However, before you begin to move applications into the Azure Infrastructure Services cloud, you might want to get some insight about network performance and your application’s network performance requirements. One helpful tool that you can use it NTttcp, which is a multithreaded, asynchronous application that sends and receives data between two or more endpoints and reports the network performance for the duration of the transfer. It is essentially a Winsock-based port of the ttcp tool that measures networking performance in terms of bytes transferred per second and CPU cycles per byte. Because it can be difficult to diagnose a system’s overall performance without dividing the system into smaller subsystems, NTttcp allows users to narrow the focus of their testing and investigation to just the networking subsystem.

In this article, Josh Adams shares will you information about the new, recently released version of NTttcp, version 5.28 – available to the public! Enjoy –Tom.

Josh AdamsLeveraged extensively by both external customers and Microsoft Product Groups alike to profile and measure Windows networking performance, updated versions of NTttcp (a Winsock-based port of BSD’s TTCP tool) have remained ‘internal-only‘ for the past five years (the last public version, v3.0, was released in 2008).  That is, of course, until now!

Download: NTttcp v5.28

I’m excited to announce that the latest version of NTttcp is now also publicly available, posted on TechNet Gallery bundled with an updated EULA.  This latest version (v5.28) brings forth both performance improvements and feature enhancements including:


  • Optimizations added for 10GigE interfaces
  • Increased platform support: x86, x64 and ARM
  • Simplified usage (e.g. sender and receiver functionality determined with a simple parameter switch)
  • Dynamic allocation of threads across all system cores (using ‘*’ for the processor ID mapping value)
  • Startup order dependency removed (i.e. server/receiver will wait for the other rather than time out)
  • Custom warm-up and cool down options available
  • Results can be outputted to XML
  • Ability to bind sender traffic to a specific IP address
  • Synchronization of data ports, useful when running multiple instances
  • An option to set the WAIT_ALL flag when using recv or WSARecv functions

Overall, the basic program workflow follows the previously released public version (v3.0), with the most notable exception related to the setup of the sender and receiver.  Rather than needing to rename the program based on function, parameters enable the selected functionality: ‘-s’  for the sender and ‘-r’ for the receiver.

For example, to measure the network performance between two multi-core serves running Windows Server 2012, NODE1 ( and NODE2 (, connected via a 10GigE connection, on NODE1 (the sender), run:

ntttcp.exe -s -m 8,*, -l 128k -a 2 -t 15

(Translation: Run ntttcp.exe as a sender, with eight threads dynamically allocated across all cores targeting, allocating a 128K buffer length and operating in asynchronous mode with 2 posted send overlapped buffers per thread for 15 seconds.)

And on NODE2 (the receiver), run:

ntttcp.exe -r -m 8,*, -rb 2M -a 16 -t 15

(Translation: Run ntttcp.exe as a receiver, with eight threads dynamically allocated across all cores listening on, allocating 64KB buffers [since -l is not specified], a 2MB SO_RCVBUF Winsock buffer and operating in asynchronous mode with 16 posted receive overlapped buffers per thread for 15 seconds.)

Using the above parameters, the program returns results on both the sender and receiver nodes, correlating network communication to CPU utilization.  Example sender-side output from a given run (which showcases a fully saturated 10GigE connection at 1131.4 MB/s):

D:\>NTttcp.exe -s -m 8,*, -l 128k -a 2 -t 15
Copyright Version 5.28
Network activity progressing…

Thread  Time(s) Throughput(KB/s) Avg B / Compl
======  ======= ================ =============
     0   30.016        77074.627    131072.000
     1   30.016       231692.964    131072.000
     2   30.016       115782.516    131072.000
     3   30.016       231496.802    131072.000
     4   30.016       116072.495    131072.000
     5   30.016        77155.650    131072.000
     6   30.016       231611.940    131072.000
     7   30.016        77633.262    131072.000

#####  Totals:  #####

   Bytes(MEG)    realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
    33959.125000      30.016       1456.355         1131.367

Throughput(Buffers/s) Cycles/Byte       Buffers
===================== =========== =============
             9050.939       2.453    271673.000

DPCs(count/s) Pkts(num/DPC)   Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
    40284.215         1.228       51434.235          0.962

Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
    24450576          1485400           0      0      4.559

And here is a Task Manager screenshot profiling the run:

Note that in keeping to the theme of making this latest version’s operation more robust, dynamic and streamlined, the following options are no longer available and/or replaced with new functionality:

-x   [PacketArray size]    
-i   Infinite Loop         
-f   <File Name>           
-fr  Full buffers posted on reads
-mb  Multiple buffer post mode

For more information about specific parameters and options, please reference the documentation included with the previous version and/or run the tool without arguments to display all supported options.  And stay tuned for future blog posts detailing the expanded functionality and additional use cases available with this latest version.

Download NTttcp v5.28 today!

Josh Adams
Senior Program Manager
Windows Server & System Center



Tom Shinder
Principal Knowledge Engineer, SCD iX Solutions Group
Follow me on Twitter: https://twitter.com/tshinder

Go Social with Building Clouds!
Building Clouds blog
Private Cloud Architecture Facebook page
Private Cloud Architecture Twitter account
Building Clouds Twitter account
Private Cloud Architecture LinkedIn Group
Building Clouds Google+ Community
Cloud TechNet forums
TechNet Cloud and Datacenter Solutions Site
Cloud and Datacenter Solutions on the TechNet Wiki