I have an extremely odd situation with BPF expressions and VLAN tagging when using ixgbe devices on CentOS 6.
I have many other CentOS 6 boxes with igb devices and I'm successfully using complex BPF expressions to match VLAN IDs and port numbers, for example:
vlan and ( ether[14:2] == 0x0005 and port 25 || ether[14:2] == 0x0006 and port 25 )
Which means: Look for packets that are VLAN tagged, where the VLAN ID is 5 and the transport protocol port number is 25, or VLAN ID is 6 and the port is 25. The is necessary due to the way BPF works, which is it will shift the byte match by 4 bytes after each occurrence of "vlan" in the expression (hence you cannot construct an expression like (vlan 5 and port 25 || vlan 6 and port 25), because the second half of the expression would only match if traffic for VLAN 6 was encapsulated inside VLAN 5). Statically using ether[a:b] will always look a-bytes deep in the packet (starting at 0) and read b-number of bytes to match. This is supposed to work regardless of whether "vlan" has occurred in the expression (making it a reliable way to find the outer VLAN ID).
So here's the problem: It doesn't work on ixgbe (at least not in promiscuous mode). When I try to match on ether[15] (the low-order bytes of the VLAN ID) it will actually match on byte 19 (the 20th byte), which mean it's shifted 4 bytes over. If I try to match on ether[11], the expression returns true when *both* byte 11 (12) AND byte 15 (16) equal the expression, which is totally bizarre. I cannot seem to make a 2-byte pattern match at all (but maybe I just didn't run tcpdump for long enough for that amazing coincidence).
By the way, I can match any other bytes normally with ether[a:b] expressions, it's only the 12-15 bytes (VLAN ethertype and VLAN ID) that have bizarre behavior.
I strongly suspect this is due to rx_vlan_offload being enabled, but when I try to disable it with ethtool I get:
$ sudo ethtool -K eth1 rx-vlan-offload off
ethtool: bad command line argument(s)
For more information run ethtool -h
This happens with both ixgbe driver version: 4.2.1-k (shipped with the CentOS kernel package), and also with version: 4.4.6 (built from source).
I found a reference to an extremely similar sounding bug here https://sourceforge.net/p/e1000/bugs/375/, but that seems to be a much earlier version of the driver. This one appears to check for RHEL_RELEASE_VERSION > 6.1 and enable 802.1P support accordingly:
#if (RHEL_RELEASE_CODE && RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(6,1))
#define HAVE_8021P_SUPPORT
#endif
So two questions:
1. How can I resolve this issue of not being able to use ether[14:2] to match VLAN ID in BPF?
2. Why can't I disable rx_vlan_offload with ethtool? (it does not say [fixed])