I have the same problem with kernel 4.4.77, ixgbe driver version 5.2.1 and ixgbevf 4.2.1. OS ALT Linux.

Spoof cheking was disabled, but no ping in two VM with the same vlan.

dmesg host:

[162311.173561] ixgbe 0000:05:00.1 eth0: 2 Spoofed packets detected

[162313.177679] ixgbe 0000:05:00.1 eth0: 1 Spoofed packets detected

[162315.181748] ixgbe 0000:05:00.1 eth0: 2 Spoofed packets detected

[162333.211099] ixgbe 0000:05:00.1 eth0: 1 Spoofed packets detected

[162337.217085] ixgbe 0000:05:00.1 eth0: 1 Spoofed packets detected

[162339.220074] ixgbe 0000:05:00.1 eth0: 2 Spoofed packets detected

# ip li show eth0

7: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000

    link/ether a0:36:9f:25:80:5e brd ff:ff:ff:ff:ff:ff

    vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 1 MAC da:55:a4:db:0f:d5, spoof checking off, link-state auto, trust off, query_rss off

    vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 3 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 4 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 5 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 6 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 7 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 8 MAC 9a:1f:86:df:b1:d8, spoof checking off, link-state auto, trust off, query_rss off

    vf 9 MAC aa:b7:85:e1:1b:06, spoof checking off, link-state auto, trust off, query_rss off

    vf 10 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

    vf 11 MAC 9a:1f:86:df:b1:d8, spoof checking off, link-state auto, trust off, query_rss off

    vf 12 MAC aa:b7:85:e1:1b:06, spoof checking off, link-state auto, trust off, query_rss off

    vf 13 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off

Hi all,

I am facing some -I think- strange problem with IPV6 and QinQ in a Linux host, and may be someone has faced similar problem (or could provide a hint).

I have some VMs running in a host (KVM), and every time any VM sends an ICMP6 Router Advertisement, we get the following log in syslog:

Aug 10 11:18:36 Hostname kernel: [1722430.045240] IPv6 header not found

For the traffic, I use QinQ (802.1Q in both tags), the inner tag is set with OVS in the tap(or bridge, no difference), and the outer is set with a veth of vlan type, in the following way:

+-------------------------------------------------------+

|                   +-----------+               HOST    |

|                   |           |         Unbuntu 16.04 |

|                   |    VM-1   |         4.4.0-62-generic

|                   |           |                       |

|                   +----+------+                       |

|                        |                              |

|                        |TAG=1                         |

|              +-----------------------+                |

|              |       OVS             |                |

|              +---------+-------------+                |

|                        | veth1.203                    |

|                        |                              |

|                        |                              |

|                        +veth0                         |

|              +---------------------+                  |

|              |      Bridge         |                  |

|              +-------+-------------+                  |

|                      |                                |

|                      |                                |

|                   +--+----+                           |

|                   |ens11f1|                           |

+-------------------------------------------------------+

'Regular traffic' (non ICMP6) seems to work fine, the problem happens apparently only with the Router Advertisement or Neighbour Discovery.

I checked the code writing that log, and I think it's in 'kernel/net/ipv6/exthdrs_core.c'

if (*offset) {

  struct ipv6hdr _ip6, *ip6;

  ip6 = skb_header_pointer(skb, *offset, sizeof(_ip6), &_ip6);

  if (!ip6 || (ip6->version != 6)) {

    printk(KERN_ERR "IPv6 header not found\n");

    return -EBADMSG;

  }

  start = *offset + sizeof(struct ipv6hdr);

  nexthdr = ip6->nexthdr;

}

but both the protocol and protocol version seem right in tcpdump:

11:28:38.675686 02:00:40:00:21:31 > 33:33:00:00:00:01, ethertype 802.1Q (0x8100), length 158: vlan 203, p 0, ethertype 802.1Q, vlan 49, p 0, ethertype IPv6, fe80::40ff:fe00:2131 > ff02::1: ICMP6, router advertisement, length 96

`....`:...........@...!1................... @...............@.!1..........@.... ........*.. ..!1................*.. ..!1............. '.

11:28:39.300076 02:00:40:00:23:2a > 33:33:00:00:00:01, ethertype 802.1Q (0x8100), length 158: vlan 204, p 0, ethertype 802.1Q, vlan 193, p 0, ethertype IPv6, fe80::40ff:fe00:232a > ff02::1: ICMP6, router advertisement, length 96

`....`:...........@...#*...................%@...............@.#*..........@.... ........*.. ..#*................*.. ..#*.............

I already disabled (just in case) all the offloading features in the ethernet card (Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)).

sysadmin@olnmpep02318n002:~/andres$ ethtool -k enp2s0f1

Features for enp2s0f1:

rx-checksumming: off

tx-checksumming: off

  tx-checksum-ipv4: off

  tx-checksum-ip-generic: off [fixed]

  tx-checksum-ipv6: off

  tx-checksum-fcoe-crc: off [fixed]

  tx-checksum-sctp: off

scatter-gather: off

  tx-scatter-gather: off

  tx-scatter-gather-fraglist: off [fixed]

tcp-segmentation-offload: off

  tx-tcp-segmentation: off

  tx-tcp-ecn-segmentation: off

  tx-tcp6-segmentation: off

udp-fragmentation-offload: off [fixed]

generic-segmentation-offload: off [requested on]

generic-receive-offload: on

large-receive-offload: off [fixed]

rx-vlan-offload: off

tx-vlan-offload: off

ntuple-filters: on

receive-hashing: on

highdma: on

rx-vlan-filter: off

vlan-challenged: off [fixed]

tx-lockless: off [fixed]

netns-local: off [fixed]

tx-gso-robust: off [fixed]

tx-fcoe-segmentation: off [fixed]

tx-gre-segmentation: on

tx-ipip-segmentation: off [fixed]

tx-sit-segmentation: off [fixed]

tx-udp_tnl-segmentation: on

fcoe-mtu: off [fixed]

tx-nocache-copy: off

loopback: off [fixed]

rx-fcs: off [fixed]

rx-all: off [fixed]

tx-vlan-stag-hw-insert: off [fixed]

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: off [fixed]

l2-fwd-offload: off [fixed]

busy-poll: off [fixed]

hw-tc-offload: off [fixed]

Any help/hint is trully appreciated!

Regards,

Andrés

Hello,

we're having problems with an Intel X710-DA4 retail card on VMware ESXi 6.5u1. After some time (usually minutes to hours) of sustained traffic on the NIC, we're seeing the following in vmkernel.log:

2017-08-11T12:26:02.554Z cpu18:66233)i40en: i40en_HandleMddEvent:6495: Malicious Driver Detection event 0x02 on TX queue 0 PF number 0x03 VF number 0x00

2017-08-11T12:26:02.554Z cpu18:66233)i40en: i40en_HandleMddEvent:6521: TX driver issue detected, PF reset issued

The network port in question is then apparently shut down, although the link stays up, and it does not pass any more network traffic. Only a reboot of the server will reset the network port and allow traffic to flow through it again.

The traffic pattern that leads to that issue usually is TCP traffic of >300MBit/s passing through a firewall virtual machine, entering on one virtual interface and exiting through another.

We are using ESXi 6.5u1 with the built-in i40en driver, as well as the latest NVM firmware version 5.05:

000:82:00.0 8086:1572 8086:0004 vmkernel vmnic2

0000:82:00.1 8086:1572 8086:0000 vmkernel vmnic3

0000:82:00.2 8086:1572 8086:0000 vmkernel vmnic4

0000:82:00.3 8086:1572 8086:0000 vmkernel vmnic5

esxcli network nic get -n vmnic3

   Advertised Auto Negotiation: false

   Advertised Link Modes: 10000BaseSR/Full

   Auto Negotiation: false

   Cable Type: FIBRE

   Current Message Level: -1

   Driver Info:

         Bus Info: 0000:82:00:1

         Driver: i40en

         Firmware Version: 5.05 0x80002898 1.1568.0

         Version: 1.3.1

   Link Detected: true

   Link Status: Up

   Name: vmnic3

   (...)

More details to curtail the problem:

• We are not using SR-IOV.
• The exact driver version is i40en 1.3.1-5vmw.650.1.26.5969303. We have observed the same issue with a previous driver version 1.3.1-1OEM.600.0.0.2768847.
• The issue happens on multiple hosts, all with the same Intel X710-DA4 adapter.

VMware Support has not been able to resolve the issue for us, saying they have been observing issues with all current X710 drivers and cannot point us into any specific direction - other than asking us to turn to Intel for support.

Honestly, at this point we're at our wits end and do not know how to proceed any further - other than switching to a different manufacturer's network hardware altogether.

Thank you for any helpful advice.

Hi,

I am also encountering this issue (4-port card).

X710/X557-AT 10GBASE-T :  FW: 5.05

Any ideas? Can the feature be disabled in the driver?

I can't even get a VM to boot when I use the i40en driver v1.3.1 under ESX v6.0u2. As soon as I power on a VM the system crashes with Malicious Driver Detection and all traffic stops.

I've had to fall back to using the i40e v2.0.6 (https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI60-INTEL-I40E-206&productId=491).

Just as you said, with v1.3.1 any decent amount of network traffic can  trigger this issue which stops ALL network traffic and requires a reboot.

2017-08-11T23:59:52.735Z cpu38:33417)i40en: i40en_HandleMddEvent:6484: Malicious Driver Detection event 0x01 on TX queue 1 PF number 0x02 VF number 0x1e

2017-08-11T23:59:52.735Z cpu38:33417)i40en: i40en_HandleMddEvent:6510: TX driver issue detected, PF reset issued

2017-08-12T00:00:00.235Z cpu38:33417)i40en: i40en_HandleMddEvent:6484: Malicious Driver Detection event 0x02 on TX queue 0 PF number 0x02 VF number 0x00

2017-08-12T00:00:00.235Z cpu38:33417)i40en: i40en_HandleMddEvent:6510: TX driver issue detected, PF reset issued

With v2.0.6, traffic hiccups but keeps flowing as soon as the driver resets (>1 sec) which usually doesn't cause an issue. This usually occurs about 100x a day across my 8 node VMware Cluster.

I do have occasions where the (TX driver issue detected, PF reset issued) occurs continuously and then it ends up causing an outage.

2017-05-26T16:01:05.347Z cpu11:33354)<6>i40e 0000:05:00.2: TX driver issue detected, PF reset issued

2017-05-26T16:01:05.538Z cpu38:33367)<6>i40e 0000:05:00.2: i40e_open: Registering netqueue ops

2017-05-26T16:01:05.547Z cpu38:33367)IntrCookie: 1915: cookie 0x38 moduleID 4111 <i40e-vmnic4-TxRx-0> exclusive, flags 0x25

2017-05-26T16:01:05.556Z cpu38:33367)IntrCookie: 1915: cookie 0x39 moduleID 4111 <i40e-vmnic4-TxRx-1> exclusive, flags 0x25

2017-05-26T16:01:05.566Z cpu38:33367)IntrCookie: 1915: cookie 0x3a moduleID 4111 <i40e-vmnic4-TxRx-2> exclusive, flags 0x25

2017-05-26T16:01:05.575Z cpu38:33367)IntrCookie: 1915: cookie 0x3b moduleID 4111 <i40e-vmnic4-TxRx-3> exclusive, flags 0x25

2017-05-26T16:01:05.585Z cpu38:33367)IntrCookie: 1915: cookie 0x3c moduleID 4111 <i40e-vmnic4-TxRx-4> exclusive, flags 0x25

2017-05-26T16:01:05.594Z cpu38:33367)IntrCookie: 1915: cookie 0x3d moduleID 4111 <i40e-vmnic4-TxRx-5> exclusive, flags 0x25

2017-05-26T16:01:05.604Z cpu38:33367)IntrCookie: 1915: cookie 0x3e moduleID 4111 <i40e-vmnic4-TxRx-6> exclusive, flags 0x25

2017-05-26T16:01:05.613Z cpu38:33367)IntrCookie: 1915: cookie 0x3f moduleID 4111 <i40e-vmnic4-TxRx-7> exclusive, flags 0x25

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 1 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 2 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 3 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 4 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 5 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 6 not allocated

2017-05-26T16:01:05.659Z cpu26:32886)<6>i40e 0000:05:00.2: Tx netqueue 7 not allocated

2017-05-26T16:01:05.660Z cpu26:32886)<6>i40e 0000:05:00.2: Netqueue features supported: QueuePair  Latency Dynamic Pre-Emptible

2017-05-26T16:01:05.660Z cpu26:32886)<6>i40e 0000:05:00.2: Supporting next generation VLANMACADDR filter

Intel Support has not been helpful in resolving these issues. They suggested disabling TSO/LRO but that didn't make a noticeable difference.

Maybe one day Intel will take the VMware i40e/i40en driver issues seriously and attempt to fix them. I've been dealing with this for 2+ years with no end insight.

Hello,

In the past 2 weeks I have gotten a hard crash on my Windows 10 (Creator's Update) PC.  WhoCrashed both times identified the intel ethernet driver as the culprit (please see below).rte

Please note that this started happening after I purchased a Western Digital NAS, which is connected by ethernet only.  I do use their backup software, but I don't think they installed any drivers.  Perhaps this is just a coincidence, or perhaps the volume of data running through the NIC has changed (I doubt this though).

I believe I already have the latest drivers.

Below you'll find the report provided by WhoCrashed.

Here are the links to the minidumps: Dropbox - 080317-16234-01.dmp  and Dropbox - 082017-16312-01.dmp

On Sun 8/20/2017 11:01:12 AM your computer crashed

crash dump file: C:\WINDOWS\Minidump\082017-16312-01.dmp
uptime: 4 days, 01:10:10
This was probably caused by the following module: e1d65x64.sys (e1d65x64+0x17407)
Bugcheck code: 0xA (0x10, 0x2, 0x0, 0xFFFFF8028B7B4073)
Error: IRQL_NOT_LESS_OR_EQUAL
file path: C:\WINDOWS\system32\drivers\e1d65x64.sys
product: Intel(R) Gigabit Adapter
company: Intel Corporation
description: Intel(R) Gigabit Adapter NDIS 6.x driver
Bug check description: This indicates that Microsoft Windows or a kernel-mode driver accessed paged memory at DISPATCH_LEVEL or above.
This appears to be a typical software driver bug and is not likely to be caused by a hardware problem.
A third party driver was identified as the probable root cause of this system error. It is suggested you look for an update for the following driver: e1d65x64.sys (Intel(R) Gigabit Adapter NDIS 6.x driver, Intel Corporation).
Google query: Intel Corporation IRQL_NOT_LESS_OR_EQUAL

On Thu 8/3/2017 11:34:32 AM your computer crashed
crash dump file: C:\WINDOWS\Minidump\080317-16234-01.dmp
uptime: 6 days, 22:29:27
This was probably caused by the following module: e1d65x64.sys (e1d65x64+0x16483)
Bugcheck code: 0x1E (0xFFFFFFFFC000001D, 0xFFFFF80E50E3FF49, 0xFFFFE60023B12E98, 0xFFFFE60023B126E0)
Error: KMODE_EXCEPTION_NOT_HANDLED
file path: C:\WINDOWS\system32\drivers\e1d65x64.sys
product: Intel(R) Gigabit Adapter
company: Intel Corporation
description: Intel(R) Gigabit Adapter NDIS 6.x driver
Bug check description: This indicates that a kernel-mode program generated an exception which the error handler did not catch.
This appears to be a typical software driver bug and is not likely to be caused by a hardware problem.
A third party driver was identified as the probable root cause of this system error. It is suggested you look for an update for the following driver: e1d65x64.sys (Intel(R) Gigabit Adapter NDIS 6.x driver, Intel Corporation).
Google query: Intel Corporation KMODE_EXCEPTION_NOT_HANDLED

Here is information about my PC:

         Operating System: Windows 10 Pro 64-bit (10.0, Build 15063) (15063.rs2_release.170317-1834)

                 Language: English (Regional Setting: English)

      System Manufacturer: ASUS

             System Model: All Series

                     BIOS: 3701

                Processor: Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz (16 CPUs), ~3.0GHz

                   Memory: 65536MB RAM

      Available OS Memory: 65444MB RAM

                Page File: 13769MB used, 61403MB available

              Windows Dir: C:\WINDOWS

          DirectX Version: DirectX 12

      DX Setup Parameters: Not found

         User DPI Setting: Using System DPI

       System DPI Setting: 96 DPI (100 percent)

          DWM DPI Scaling: Disabled

                 Miracast: Available, with HDCP

Microsoft Graphics Hybrid: Not Supported

           DxDiag Version: 10.00.15063.0000 64bit Unicode

The motherboard is an ASUS X99 Deluxe

Hi Sharon,

I have the same problem.

1. FreeBSD 11.1-RELEASE

2. ixl0: <Intel(R) Ethernet Connection XL710/X722 Driver, Version - 1.7.12-k>

ixl0: fw 5.0.40043 api 1.5 nvm 5.04 etid 80002505 oem 0.0.0

3. this is iscsi target server. If traffic with high load is flowing on the server, error log will occur frequently.

ixl0: WARNING: queue 2 appears to be hung!

ixl0: WARNING: Resetting!

ixl0: Malicious Driver Detection event 2 on TX queue 6, pf number 0

ixl0: MDD TX event is for this function!

It does not solve even driver version 1.7

Regards,

Takunii

Hi, I have several X710-DA4 that I purchased at different times, and some of them I was able to grab the latest firmware (5.05) and upgrade them. nvmupdate64e and ethool show this on the good ones:

driver: i40e

version: 1.6.42

firmware-version: 5.05 0x8000289d 1.1568.0

bus-info: 0000:85:00.2

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.

Inventory in progress. Please wait [.........*]

Num Description                               Ver. DevId S:B    Status

=== ======================================== ===== ===== ====== ===============

01) Intel(R) Ethernet Converged Network       5.05  1572 00:004 Up to date

    Adapter X710-4

02) Intel(R) I350 Gigabit Network Connection  1.99  1521 00:129 Update not

                                                                available

03) Intel(R) Ethernet Converged Network       5.05  1572 00:133 Up to date

    Adapter X710-4

On the other box, it will not let me upgrade:

driver: i40e

version: 2.0.23

firmware-version: 4.10 0x800011c5 0.0.0

bus-info: 0000:01:00.1

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.

Inventory in progress. Please wait [|.........]

Num Description                               Ver. DevId S:B    Status

=== ======================================== ===== ===== ====== ===============

01) Intel(R) Ethernet Converged Network       4.10  1572 00:001 Update not

    Adapter X710-4                                              available

02) Intel(R) I350 Gigabit Network Connection  1.99  1521 00:129 Update not

                                                                available

03) Intel(R) Ethernet Converged Network       4.10  1572 00:130 Update not

    Adapter X710-4                                              available

Does anyone know what's wrong?

Hello,

I am new to this community. I subscribed to ask a question here because I am really stuck.

My company makes electronic devices that use UDP 100 MB Ethernet communication with a PC (under Windows 7, 8.1 or 10).

It works perfectly with other brands of Ethernet adapters; but there is a strange behavious with the Intel NICs we tried (at least I219-V, I219-LM and I217-LM).

Basically, our electronic devices can be considered as cameras capturing about 150 images per second.

We send a small command on one socket via UDP to tell it to capture an image, then we receive the compressed image as a set of UDP packets on another socket.

Each packet contains up to 1444 bytes of data (to which one can add the data from the different layers of protocols, which in the end does not exceed the standard packet size => no need to use Jumbo packets).

The problem is that, sometimes (this varies from as often as every 5 seconds to as rarely as only once within a 10-mn period), I am waiting forever (until the defined UDP time out) for the data to arrive while it has been sent (I can see it by sniffing the data from another computer connected to the same switch). I could believe that the UDP packet was lost by the Intel NIC, but it has not been lost. If I send a new image capture command, the packets that I was waiting finally arrive, followed by the packets of the new image.

Why are those packets stuck?

Is there any advanced parameter that I could modify from the device driver's configuration window or from a Registry key?

Note that I tried updating the Intel driver to recent versions (last one is 22.4.0.1). It seems to perform a little better that some other versions I tried (like the one installed by default in Windows), but none of the versions I tried work perfectly.

Many thanks in advance if you can help me understand what is wrong (either from me or from the driver). I have been struggling on this problem for months and we have to equip our customers who own a laptop with an Intel NIC with USB adapters with a NIC made by another brand to bypass the problem.

Karl

Hoping someone can help.

Recently upgraded a server running Xen from 3.18.44-20.el6.x86_64 to 4.9.39-29.el6.x86_64 (did a yum update so other packages would have been updated to).

Since rebooting, we randomly get "e1000e: eth0 NIC Link is Down", this happens for only a second, but since we have this interface bonded it can be enough for it to bring down the interface fail over, and then come back again, which causes packet loss.

If tried turning pcie_aspm off, flow control off, nothing seems to help. The problem seems to be worse when high traffic is being used, but also happens randomly as well.

NIC Intel Corporation 82574L Gigabit on a Supermicro box. Driver 3.2.6-k

Have now upgraded to 3.3.5.10, the problem seems to be happening a lot less often, but I'm not sure if its a coincidence.

I work in IT at a corporation where almost all of our current computers have the Intel Ethernet Connection l218-LM network adapters. I have seen 5 computers in the last 2 months come up with the error where this network suddenly stops working and produces Error Code 10: this device cannot start. I have searched high and low for a solution, and the ULP enable/disable utility is the only thing I haven't tried yet. Can someone please send the utility to me?

Hello ,

We are using an X520 intel 10Gbps dual nic.

It always returning this kind of errors and causing big latencies on the network at the up / down times . This especially happens under high load .

Thank you for your help

[14171.058661] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14172.839458] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14341.651931] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14343.017301] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14425.865861] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14427.179247] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14505.831406] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14507.244369] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14512.896823] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14513.580471] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14533.355178] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14534.252857] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14618.865777] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14620.463488] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14662.487481] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14662.815243] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[14684.846735] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[14685.199573] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15044.844396] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15045.318274] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15059.158149] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15059.894558] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15093.595264] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15094.903176] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15095.079577] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15095.215190] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15177.123291] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15177.856691] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15180.633684] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15182.136821] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15182.344764] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15182.864839] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15188.694321] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15189.480957] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15211.220648] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15212.089333] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15217.036162] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15218.554021] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[15237.770249] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[15239.225842] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[16186.786363] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[16187.659374] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[16775.010117] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[16775.638209] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[16825.296468] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[16827.031402] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[16840.625743] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[16841.559462] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18408.471280] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18408.812374] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18431.637213] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18433.590395] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18436.023824] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18437.877135] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18843.605181] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18845.628759] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18889.130249] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18890.877972] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18891.138043] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18893.117385] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18981.836999] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18982.294955] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18983.713804] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18984.046968] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[18984.203464] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[18984.463010] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19021.600243] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19021.959682] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19230.325644] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19230.595523] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19234.591401] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19234.883580] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19342.459429] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19343.621629] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19369.107665] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19370.438448] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19378.087075] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19378.823233] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19386.175615] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19387.206941] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19402.015296] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19403.783442] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19425.428238] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19426.503166] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19435.364329] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19436.935573] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19442.782976] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19443.271474] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19464.773444] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19465.992381] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19534.022782] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19535.817291] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19819.855723] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19820.318408] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[19975.257266] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[19976.529280] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22620.549731] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22620.738098] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22720.159576] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22721.515905] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22721.829413] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22722.675934] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22908.480812] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22908.951434] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22909.838174] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22910.208957] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22911.991401] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22912.303507] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22912.777158] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22914.535515] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22915.704775] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22916.935550] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22917.563142] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22918.095609] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22919.877803] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22921.544138] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22923.371469] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22923.783676] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22923.912254] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22924.199720] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22924.760769] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22926.471778] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22926.575668] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22926.679744] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22929.204169] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22930.951812] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22989.470222] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22990.536931] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[22993.584348] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[22994.825011] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23008.916944] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23009.353308] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23032.423160] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23034.121738] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23036.280618] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23036.465760] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23050.048766] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23051.746073] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23052.058021] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23052.578081] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23101.281013] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23101.898993] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23128.301782] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23128.715464] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23136.469441] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23137.099606] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23141.687157] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23143.563746] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23162.527432] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23164.236437] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23173.189974] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23174.668328] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23207.450333] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23207.732953] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23215.809467] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23216.141058] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23223.154873] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23224.773212] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23271.998327] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23272.118129] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23660.993828] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23662.189331] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23662.468246] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23665.589412] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23665.653323] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23666.069387] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23673.275492] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23674.765504] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23700.290010] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23701.782014] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23722.454414] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23723.294442] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23723.950353] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23724.262413] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23724.463716] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23726.486661] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23740.014452] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23741.158798] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23741.222741] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23741.430802] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23742.243973] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23743.702866] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23751.703578] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23752.086958] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23757.096288] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23757.559098] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23778.029984] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23779.095454] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23799.768859] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23801.815851] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23810.517798] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23812.248042] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23816.398680] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23816.744151] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23878.946295] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23880.913355] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23985.476117] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23985.771325] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23987.085000] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23987.995347] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[23998.666069] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[23999.027534] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24008.216386] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24008.859691] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24011.953851] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24013.251808] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24016.403773] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24017.875905] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24018.070427] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24018.291924] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24114.842246] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24115.197629] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24576.995336] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24577.766245] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24621.070982] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24621.175034] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24621.869296] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24623.399034] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24669.005148] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24670.191937] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24672.841196] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24672.935963] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24696.406685] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24697.704465] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24698.650967] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24699.944479] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24722.964115] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24724.713121] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24727.939431] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24728.337019] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24733.653890] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24735.753148] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24735.884151] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24735.961149] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24736.065098] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24736.169152] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24739.753395] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24740.561247] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24744.312840] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24744.873322] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24744.977240] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24745.705362] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24747.105016] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24748.265388] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24750.811017] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24752.553646] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24770.391944] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24771.177798] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24782.275169] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24783.658162] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24797.232197] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24798.186300] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[24981.679445] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[24982.701767] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[27212.732410] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[27213.206885] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[27656.488814] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[27658.015502] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[27740.012810] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[27740.232543] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[27929.654426] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[27930.917379] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28212.980410] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28213.737352] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28215.192322] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28215.977829] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28349.224289] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28350.371883] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28362.792631] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28362.924052] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28364.974366] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28365.164141] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28367.475189] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28369.452342] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28419.143287] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28420.846841] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28440.246831] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28441.501524] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28441.765401] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28442.229549] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28442.408613] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28443.325518] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28444.405502] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28445.870248] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28457.559424] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28458.093820] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28458.748116] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28459.213885] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28510.785219] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28511.983234] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28568.902953] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28569.519921] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28666.715422] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28668.017886] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28680.221951] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28680.601948] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28717.935982] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28719.730666] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28734.236906] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28734.466956] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[28767.101871] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[28767.411535] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[30923.792843] perf: interrupt took too long (3917 > 3913), lowering kernel.perf_event_max_sample_rate to 51000

[31401.713468] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[31403.428609] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[32671.459579] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[32671.667647] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[32677.341644] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[32678.011764] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[32719.235190] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[32721.212546] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[32810.415467] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[32811.518169] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33191.817707] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33192.645270] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33256.920521] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33258.670458] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33264.839522] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33265.030520] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33269.917816] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33271.366877] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33273.165648] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33273.606759] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33273.898768] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33275.846791] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33283.995586] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33284.230927] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33308.541268] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33308.999357] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33313.318040] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33315.335817] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33319.798982] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33321.671959] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33943.433762] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33944.467160] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[33949.425445] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[33950.803772] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[34781.379419] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[34782.499984] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[34935.326852] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[34936.613748] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[34986.162214] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[34988.006447] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35221.850634] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35223.723083] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35288.108704] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35289.451926] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35554.698576] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35555.889024] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35568.675523] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35570.417297] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35635.255165] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35636.146349] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35638.019273] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35638.386376] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[35648.011408] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[35648.818594] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36033.667888] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36034.121693] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36153.212995] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36153.571924] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36157.421213] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36158.307991] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36159.022709] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36159.172006] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36193.869222] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36194.236616] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36314.566373] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36316.414925] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

[36363.949603] ixgbe 0000:04:00.1 p1p2: NIC Link is Down

[36365.759790] ixgbe 0000:04:00.1 p1p2: NIC Link is Up 10 Gbps, Flow Control: None

Hi Sharon and Pratik,

I have the exact same need - I have an 82599ES-based SR-IOV setup using a Mirantis Fuel 9.2 OpenStack deployment where I am trying to pass VLAN-tagged packets out of a KVM VM instance with the VLAN kept intact.  Just like Pratik, I can work with a situation where either the tag applied by the virtual machine is retained with no tag added to it on egress or (preferably) a situation where the inner tag applied by the VM is preserved and an outer tag is added by the network adapter.

I have a full lab environment set up to test this and I will be compiling both drivers. However, I'm curious if there is a specific fix that addresses this issue and if you have any additional information on what behavior I should expect. Will the ixgbe 5.1 driver and the ixgbevf 4.1 driver allow both of the situations I described, i.e. either one tag or double tags leaving the VM?  Thank you for any clarification you can provide,

A.C.

******

I have looked, but had been unable to locate the answer to this most basic of questions regarding these popular Intel gigabit fiber server NICs.    The i710 10G fiber NIC supports several different Intel branded multi-speed tranceivers, but no information is available for it's cousin.

Hello,

After a clear installation of Windows 10 64bit I am experiencing frequent disconnecting my internet ethernet connection, especially after awaking from sleep mode.

After rebooting the system the internet connection is functioning again.

I tried to install the newest driver from intel download site more times without success:

Download Intel® Network Adapter Driver for Windows® 10*

My Motherboard: MSI-7760

Failure protocol attached.

I read about a patch which could solve the issue.

Thank you very much for your support!

Vojtech

I've been trying to troubleshoot this issue now for months on and off and have gotten nowhere.

It seems that WOL is broken when using this hardware under Windows 10.

I've been using the Windows 10 driver that installs by default (version 12.155.22.6 dated 4/5/2016) and have not been able to find a different one (latest ProSet software doesn't contain a driver for this adaptor).

I've set everything up that various internet threads I've read have said to do, this includes:

BIOS:

Enable the adaptor (duh)

Enable Remote Wake Up

Disable Low Power Mode

Windows:

Disable Fast Startup

Make sure, under the Power Management tab, that all the boxes are checked.

There are no properties for "Waking" in Advanced Properties for the adaptor so I can't set them, nor is there a PME Event Property to check/change.

---

I know that WOL is possible for this machine because I popped in a PCI NIC and got WOL to work on that NIC, but this integrated adaptor just won't comply.

Lot's of threads out there saying it can be done, but I can't seem to do it on my hardware.

Here's a capture from BOOTUTIL showing that WOL is supposed to work...

---

C:\Intel22.4.0.1\APPS\BootUtil\Winx64>BOOTUTILW64E.EXE

Intel(R) Ethernet Flash Firmware Utility
BootUtil version 1.6.40.1
Copyright (C) 2003-2017 Intel Corporation

Port   Network Address     Location    Series     WOL Flash Firmware                                     Version
==== =============== ======== ======= ===  ============================= =======
1       0024E819DC99        0:25.0       Gigabit    YES  FLASH Not Present

---

One suggestion I found was to update the boot agent (mine is 1.3.81) to the latest (1.5.something) and then throw a flag in the boot agent setup that supports WOL.

However, I do not think I can update the boot agent as BOOTUTIL says there's no flash firmware present to do so, or correct me if I'm wrong.

Ideas on how to update the boot agent?

Aside from that any other ideas?

===

Editing to add infomation

===

Well, I never thought I'd see the day that Linux supported old hardware better than Windows but that day is today.

Reference this thread: Wake on Lan, Windows 10 Pro, integrated 82567LM-3

And see what the Fabrice person says.

The instructions in this referenced thread above were too generic to work verbatim, but they were enough for me to figure my own way to try out WOL under a Linux distro lickety-split.

In a nutshell this person suggests booting to Linux, turning on WOL, then booting back to Windows, as if that would permanently and magically enable WOL in Windows.

Well, nice try, that doesn't work because, I assume, Windows initializes the adapter when it gets back in control.

What DOES work is Linux.  I was able to create a bootable Ubuntu USB stick and in under 5 minutes booted the machine into Ubuntu and tested WOL.  It WORKED. Wow.

So what we have is a crappy Windows 10 driver that doesn't support WOL, at least not with ANY combination of windows settings I've tried.

There have got to be 10s of 1000s of OptiPlex 760s (and not to mention other models with this common adapter) out in the wilds of the business world, and Microsoft and/or Intel, with this stunted driver, has basically shat all over the owners with regards to the management of these computers with these adapters.  If I was a IT guy for a company with 1000 of these and couldn't wake any up to do remote management, I'd be pretty damn grumpy.  I'm grumpy now and I only have 10 of these.

Way to go WinTel.

So, back to the original problem...anyone have any ideas on how it might be gotten to work?

Custom Driver?

Hack current driver?

New driver not commonly known?

Some setting I've missed?

Anything...anything to make this work.

Hi,

I have 2 Cars Type XL710QDA2 and I'd like to use Mellanox MAM Modules to use each port with 1x10G. NVM on both cards is 5.05. I set both cards to 4x10, but one card is not responsive after reboot anymore. How to change modes and how to get the other card back to life?

Regards,

Michael!

Versions:

NVM: 5.05

QCU v2.27.10.01

i40e: 2.1.26

./qcu64e /devices

Intel(R) QSFP+ Configuration Utility

QCU version: v2.27.10.01

Copyright(C) 2016 by Intel Corporation.

Software released under Intel Proprietary License.

NIC Seg:Bus Ven-Dev   Mode    Adapter Name

=== ======= ========= ======= ==================================================

1) 000:003 8086-1583 Error: Connection to driver failed.

2) 000:004 8086-1583 4x10    Intel(R) Ethernet Converged Network Adapter XL710-

Error: Base driver is not available for this adapter. Please ensure the driver is correctly attached to the device.

Hi,

I am using a Dell PowerEdge R730 with Dual Xeon, each 22 cores, with 6 ixgbe compatible cards, on which I am running Linux with ixgbe driver version 4.4.0-k, using kernel versions both 4.7.10 and 4.9.6.
I am loading the ixgbe modules at boot time, bringing up the interfaces and setting smp_affinity to the cards, using the set_irq_affinity script, so all the possible RxTx IRQs are distributed between all the available cores.
The problem is that it happens, random, but quite often that the smp_affinity setting fails, and I need manually to re-run the script one or more times in order desired settings to be applied. There were also several occasions when the settings were not applied at all, and it took me several reboots to script to start working again.
The problem appears not only randomly as occurrence, but also at random NIC controllers, so I am excluding the possibility of failed HW, since I also changed NICs.

I added some debug messages to track the affinity setting in Linux kernel, and it turns out that most of the times when the setting fails the error that affinity setting function irq_do_set_affinity returns is EBUSY, but also sometimes it returns ENOSPC.

More investigation on the topic showed whenever EBUSY was returned the problem could be overcome with re-running the script. But if the error returned was ENOSPC, it takes several reboots for the problem to disappear.

In order to provide some more details on the system I am attaching two text files with the output of the modinfo of the ixgbe and lspci on the machine.

Hi,

I am also encountering this issue (4-port card).

X710/X557-AT 10GBASE-T :  FW: 5.05

Any ideas? Can the feature be disabled in the driver?

hello all：

          I am receiving the follwoing error when I installed the igb-5.3.5.10

[ 2617.676535] igb 0000:04:00.0: The NVM Checksum Is Not Valid

[ 2617.702549] igb 0000:04:00.1: The NVM Checksum Is Not Valid

[ 2617.728535] igb 0000:04:00.2: The NVM Checksum Is Not Valid

[ 2617.754518] igb 0000:04:00.3: The NVM Checksum Is Not Valid

Linux version 3.10.0-327.el7.x86_64

CentOS Linux release 7.2.1511 (Core)

Please provide troubleshooting steps.

thanks.