Menu

Mellanox ConnectX-3

We recommend using the latest device driver from Mellanox rather than the one in your Linux distribution.

Note that the Mellanox device driver installation script automatically adds the following to your /etc/sysctl.conf file:

## MLXNET tuning parameters ##
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 0
net.ipv4.tcp_low_latency = 1
net.core.netdev_max_backlog = 250000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 16777216
net.core.wmem_default = 16777216
net.core.optmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
## END MLXNET ##

Be aware that this override any tuning you have in that file! While we agree with these settings for good for a LAN, for a WAN you will need much larger TCP buffers, and we recommend turning on SACK.  Use something like this instead:

## FASTERDATA 40G tuning parameters ##
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_low_latency = 0
net.core.netdev_max_backlog = 250000
net.core.rmem_max = 268435456
net.core.wmem_max = 268435456
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728 ## END FASTERDATA ##