Remote Direct Memory Access, or RDMA, allows a network device to transfer data directly to and from application memory on another system, increasing throughput and lowering latency in certain networking environments.
To avoid performance degradation from dropped packets enable link level flow control or priority flow control on all network interfaces and switches.
![]() |
NOTE: |
# tar zxf i40iw-<x.x.x>.tar.gz
# cd i40iw-<x.x.x>
# ./build.sh /opt/i40e-<x.x.x> k
# ethtool -A ethX rx <on> tx <on>
![]() | NOTE: This command only enables or disables Flow Control if auto-negotiation is disabled. If auto-negotiation is enabled, this command changes the parameters used for auto-negotiation with the link partner. |
Basic FreeBSD RDMA installation instructions:
# tar -xf ixl-<version>.tar.gz
# tar -xf iw_ixl-<version>.tar.gz
# cd ixl-<version>/src directory
# make
# make install
# cd iw_ixl-<version>/src
# make clean
# make IXL_DIR=$PATH_TO_IXL/ixl-<version>/src
# make install
# sysctl dev.ice.<interface_num>.fc=3
FreeBSD RDMA installation and configuration information is provided in the driver README file in the driver tarball for Intel Ethernet devices that support RDMA.
Network Direct (ND) allows user-mode applications to use RDMA features.
![]() |
NOTE: User mode applications may have prerequisites such as Microsoft HPC Pack or Intel MPI Library, refer to your application documentation for more details. |
The Intel® Ethernet User Mode RDMA Provider is supported on Microsoft* Windows Server* 2012 R2 and later.
Follow the steps below to install user-mode Network Direct features.
![]() | NOTE: If Windows Firewall is disabled or you are using a third party firewall, you will need to add this rule manually. |
RDMA Network Direct Kernel (NDK) functionality is included in the Intel base networking drivers and requires no additional features to be installed.
If you want to allow NDK's RDMA functionality across subnets, you will need to select "Enable RDMA routing across IP Subnets" on the RDMA Configuration Options screen during base driver installation.
You can check that RDMA is enabled on the network interfaces using the following Microsoft* PowerShell* command:
Get-NetAdapterRDMA
Use the following PowerShell command to check if the network interfaces are RDMA capable and multichannel is enabled:
Get-SmbClientNetworkInterface
Use the following PowerShell command to check if Network Direct is enabled in the operating system:
Get-NetOffloadGlobalSetting | Select NetworkDirect
Use netstat to make sure each RDMA-capable network interface has a listener at port 445 (Windows Client OSs that support RDMA may not post listeners). For example:
netstat.exe -xan | ? {$_ -match "445"}
To enable RDMA functionality on virtual adapter(s) connected to a VMSwitch, the SRIOV (Single Root IO Virtualization) and VMQ (Virtual Machine Queues) advanced properties must be enabled on each port. Under certain circumstances, these settings may be disabled by default. These options can be set manually in the advanced tab of the adapter properties dialog box, or with the following Powershell commands:
Set-NetAdapterAdvancedProperty -Name <nic_name> -RegistryKeyword *SRIOV -RegistryValue 1
Set-NetAdapterAdvancedProperty -Name <nic_name> -RegistryKeyword *VMQ -RegistryValue 1
NDK Mode 3 allows kernel mode Windows components to use RDMA features inside Hyper-V guest partitions. To enable NDK mode 3 on an Intel Ethernet device, do the following:
New-VMSwitch -Name <switch_name> -NetAdapterName <device_name>
-EnableIov $true
Set-NetAdapterAdvancedProperty -Name <device_name> -RegistryKeyword RdmaMaxVfsEnabled -RegistryValue <Value: 0 – 32>
Get-NetAdapterRdma | Disable-NetAdapter
Get-NetAdapterRdma | Enable-NetAdapter
Add-VMNetworkAdapter -VMName <vm_name> -VMNetworkAdapterName <device_name> -SwitchName <switch_name>
Set-VMNetworkAdapterRdma -VMName <vm_name> -VMNetworkAdapterName <device_name> -RdmaWeight 100
Set-VMNetworkAdapter -VMName <vm_name> -VMNetworkAdapterName <device_name> -IovWeight 100
Set-NetAdapterAdvancedProperty -Name <device_name> -RegistryKeyword RdmaVfEnabled -RegistryValue 1
Get-NetAdapterRdma | Enable-NetAdapterRdma
NDK allows Windows components (such as SMB Direct storage) to use RDMA features.
This section outlines the recommended way to test RDMA for Intel Ethernet functionality and performance on Microsoft* Windows* operating systems.
Note that since SMB Direct is a storage workload, the performance of the benchmark may be limited to the speed of the storage device rather than the network interface being tested. Intel recommends using the fastest storage possible in order to test the true capabilities of the network device(s) under test.
Test instructions:
Copyright (C) 2019, Intel Corporation. All rights reserved.
Intel Corporation assumes no responsibility for errors or omissions in this document. Nor does Intel make any commitment to update the information contained herein.
Intel is a trademark of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
This software is furnished under license and may only be used or copied in accordance with the terms of the license. The information in this manual is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Intel Corporation. Intel Corporation assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without the express written consent of Intel Corporation.