Cisco M5 server, with 6 nics on it. The first two are 1G nics that are unused.
The last 4, are:
vmnic2 - 10G nic, Intel XL710, driver version 2.1.5.0 FW version 8.50, link state up
vmnic3 - 10G nic, Intel XL710, driver version 2.1.5.0 FW version 8.50, link state up
vmnic4 - 10G nic, Intel XL710, driver version 2.1.5.0 FW version 8.50, link state up
vmnic5 - 10G nic, Intel XL710, driver version 2.1.5.0 FW version 8.50, link state up
Worth mentioning:
vmnic 2 and 4 are uplinks, using a standard Distributed Switch (virtual switch) for those uplinks.
vmnic 3 and 5 are connected to an N-VDS virtual switch (used with NSX-T) and don't have uplinks.
In ESXi (VMWare Hypervisor, v7.0), we have set the RSS values accordingly:
UPDATED: how we set the RSS Values!
first, make sure that RSS parameters are unset. Because DRSS and RSS should not be set together.
> esxcli system module parameters set -m -i40en -p RSS=""
next, make sure that DRSS parameters are set. We are setting to 4 Rx queues per relevant vmnic.
esxcli system module parameters set -m -i40en -p DRSS=4,4,4,4
now we list the parameters to ensure they took correctly
> esxcli system module parameters list -m i40en
Name Type Value Description
------------- ------------ ------- -----------
DRSS array of int Enable/disable the DefQueue RSS(default = 0 )
EEE array of int Energy Efficient Ethernet feature (EEE): 0 = disable, 1 = enable, (default = 1)
LLDP array of int Link Layer Discovery Protocol (LLDP) agent: 0 = disable, 1 = enable, (default = 1)
RSS array of int 4,4,4,4 Enable/disable the NetQueue RSS( default = 1 )
RxITR int Default RX interrupt interval (0..0xFFF), in microseconds (default = 50)
TxITR int Default TX interrupt interval (0..0xFFF), in microseconds, (default = 100)
VMDQ array of int Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default =8)
max_vfs array of int Maximum number of VFs to be enabled (0..128)
trust_all_vfs array of int Always set all VFs to trusted mode 0 = disable (default), other = enable
But, we are seeing this when we look at the individual adaptors in the ESXi kernel:
> vsish -e get /net/pNics/vmnic3/rxqueues/info
rx queues info {
# queues supported:1
# rss engines supported:0
# filters supported:0
# active filters:0
# filters moved by load balancer:0
RX filter classes: 0 -> No matching defined enum value found.
Rx Queue features: 0 -> NONE
}
Nics 3 and 5, connected to the N-VDS virtual switch, only get one single Rx Queue supported, even though the kernel module is configured properly.
> vsish -e get /net/pNics/vmnic2/rxqueues/info
rx queues info {
# queues supported:9
# rss engines supported:1
# filters supported:512
# active filters:0
# filters moved by load balancer:0
RX filter classes: 0x1f -> MAC VLAN VLAN_MAC VXLAN Geneve
Rx Queue features: 0x482 -> Pair Dynamic GenericRSS
But Nics 2 and 4, which are connected to the standard distributed switch, have 9 Rx Queues configured properly.
Is this related to the virtual switch we are connecting to (meaning we need to be looking at VMWare)? Or, is this somehow related to the i40en driver that is being used (in which case we need to be going to server vendor or Intel who makes the XL710 nic)?
