There are two blog posts explaining in great detail what you need to do to use Mellanox ConnectX-2 or ConnectX-3 cards to implement RDMA networking for SMB 3.0 (using SMB Direct). You can find them at:
- Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-3 using 10GbE/40GbE RoCE – Step by Step
- Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand – Step by Step
However, I commonly get some question where SMB cmdlets reports a Mellanox NIC as not being RDMA-capable. Over time, I learned that the most common issue around this is an outdated firmware. Windows Server 2012 now comes with an inbox driver for these Mellanox adapters, but it is possible that your firmware on the adapter itself is old. This will cause the NIC to not use RDMA.
To be clear, your Mellanox NIC must have firmware version 2.9.8350 or higher to work with SMB. The driver actually checks the firmware version on start up and logs a message if the firmware does not meet this criteria: "The firmware version that is burned on the device <device name> does not support Network Direct functionality. This may affect the File Transfer (SMB) performance. The current firmware version is <current version> while we recommend using firmware version 2.9.8350 or higher. Please burn a newer firmware and restart the Mellanox ConnectX device. For more details about firmware burning process please refer to Support information on http://mellanox.com".
However, since the NIC actually works fine without RDMA (at reduced performance and higher CPU utilization), some administrators might fail to identify this issue. If they are following the steps outlined in the links above, the verification steps will point to the fact that RDMA is actually not being used and the NIC is running only with TCP/IP.
The solution is obviously to download the firmware update tools from the Mellanox site and fix it. It will also come with the latest driver version, which is newer than the inbox driver. The direct link to that Mellanox page is http://www.mellanox.com/content/pages.php?pg=products_dyn&product_family=32&menu_section=34. You need to select the “Windows Server 2012” tab at the bottom of that page and download the "MLNX WinOF VPI for x64 platforms" package, shown in the picture below.