I am evaluating the Ethernet Server Adapter I350-T2 to replace our previous end of life card. I have not had any issues with the card during my operational tests but when I execute the cards offline tests the network interface goes offline and requires the network service to be restarted. My concern that there is some real instability between my hardware/software and the I350 that could manifest in failures over prolonged use.
Here are my test results and kernel log data.
$ sudo ethtool --test eth2 offline
Cannot get strings: No such device
2016-03-03 11:04:34.833 localhost kernel: igb 0000:01:00.0: offline testing starting
2016-03-03 11:04:34.833 localhost kernel: igb 0000:01:00.0: eth2: igb: eth2 NIC Link is Down
2016-03-03 11:04:36.508 localhost kernel: igb 0000:01:00.0: eth2: igb: eth2 NIC Link is Up 100 Mbps Full Duplex, Flow Control: None
2016-03-03 11:04:36.943 localhost avahi-daemon[948]: Interface eth2.IPv4 no longer relevant for mDNS.
2016-03-03 11:04:36.943 localhost avahi-daemon[948]: Leaving mDNS multicast group on interface eth2.IPv4 with address 192.168.50.2.
2016-03-03 11:04:36.944 localhost kernel: igb 0000:01:00.0: eth2: PCIe link lost, device now detached
2016-03-03 11:04:36.944 localhost kernel: igb 0000:01:00.0: pattern test reg 002C failed: got 0x0000FFFF expected 0x00005A5A
2016-03-03 11:04:38.621 localhost kernel: igb 0000:01:00.0: testing shared interrupt
2016-03-03 11:04:40.315 localhost kernel: igb 0000:01:00.0: Cannot do PHY loopback test when SoL/IDER is active.
2016-03-03 11:04:45.155 localhost avahi-daemon[948]: Withdrawing address record for fe80::a236:9fff:XXXX:XXXX on eth2.
2016-03-03 11:04:45.155 localhost avahi-daemon[948]: Withdrawing address record for 192.168.50.2 on eth2.
2016-03-03 11:04:45.155 localhost ntpd[2499]: Deleting interface #8 eth2, fe80::a236:9fff:XXXX:XXXX#123, interface stats: received=0, sent=0, dropped=0, active_time=6201 secs
2016-03-03 11:04:45.155 localhost ntpd[2499]: Deleting interface #4 eth2, 192.168.50.2#123, interface stats: received=0, sent=0, dropped=0, active_time=6201 secs
Why is this offline test failing? Should I even worry that this test fails? Does anybody have some tips as to what I should change/fix/do to get the test to pass?
Thanks,
Stephan
Some background details:
OS: Linux Fedora 13
Motherboard: AIMB-780 w/ Intel Core i5-660
$ sudo ethtool -i eth2
driver: igb
version: 5.3.3.5
firmware-version: 1.63, 0x80000cbb
bus-info: 0000:01:00.0
$uname -r
2.6.33.5-112.fc13.i686.PAE
$ sudo lspci -s 01:00 -nnvvvk
01:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter I350-T2 [8086:0002]
Physical Slot: 128
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 16
Region 0: Memory at fbd00000 (32-bit, non-prefetchable) [size=1M]
Region 3: Memory at fbcfc000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 unlimited, L1 <32us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-87-06-7c
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 00
VF offset: 384, stride: 4, Device ID: 1520
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 00000000c0000000 (64-bit, prefetchable)
Region 3: Memory at 00000000c0020000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] #17
Capabilities: [1c0 v1] #18
Capabilities: [1d0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: igb
Kernel modules: igb
01:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter I350-T2 [8086:0002]
Physical Slot: 128
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin B routed to IRQ 17
Region 0: Memory at fbb00000 (32-bit, non-prefetchable) [size=1M]
Region 3: Memory at fbcf8000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00002000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 unlimited, L1 <32us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-87-06-7c
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 01
VF offset: 384, stride: 4, Device ID: 1520
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 00000000c0040000 (64-bit, prefetchable)
Region 3: Memory at 00000000c0060000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] #17
Capabilities: [1d0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: igb
Kernel modules: igb
$ modinfo igb
filename: /lib/modules/2.6.33.5-112.fc13.i686.PAE/kernel/drivers/net/igb/igb.ko
version: 5.3.3.5
license: GPL
description: Intel(R) Gigabit Ethernet Network Driver
author: Intel Corporation, <e1000-devel@lists.sourceforge.net>
srcversion: F03F0C4A4602E651FFBECFA
alias: pci:v00008086d000010D6sv*sd*bc*sc*i*
alias: pci:v00008086d000010A9sv*sd*bc*sc*i*
alias: pci:v00008086d000010A7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E8sv*sd*bc*sc*i*
alias: pci:v00008086d00001526sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Dsv*sd*bc*sc*i*
alias: pci:v00008086d000010E7sv*sd*bc*sc*i*
alias: pci:v00008086d000010E6sv*sd*bc*sc*i*
alias: pci:v00008086d00001518sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Asv*sd*bc*sc*i*
alias: pci:v00008086d000010C9sv*sd*bc*sc*i*
alias: pci:v00008086d00000440sv*sd*bc*sc*i*
alias: pci:v00008086d0000043Csv*sd*bc*sc*i*
alias: pci:v00008086d0000043Asv*sd*bc*sc*i*
alias: pci:v00008086d00000438sv*sd*bc*sc*i*
alias: pci:v00008086d00001516sv*sd*bc*sc*i*
alias: pci:v00008086d00001511sv*sd*bc*sc*i*
alias: pci:v00008086d00001510sv*sd*bc*sc*i*
alias: pci:v00008086d00001527sv*sd*bc*sc*i*
alias: pci:v00008086d0000150Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000150Esv*sd*bc*sc*i*
alias: pci:v00008086d00001524sv*sd*bc*sc*i*
alias: pci:v00008086d00001523sv*sd*bc*sc*i*
alias: pci:v00008086d00001522sv*sd*bc*sc*i*
alias: pci:v00008086d00001521sv*sd*bc*sc*i*
alias: pci:v00008086d00001539sv*sd*bc*sc*i*
alias: pci:v00008086d0000157Csv*sd*bc*sc*i*
alias: pci:v00008086d0000157Bsv*sd*bc*sc*i*
alias: pci:v00008086d00001538sv*sd*bc*sc*i*
alias: pci:v00008086d00001537sv*sd*bc*sc*i*
alias: pci:v00008086d00001536sv*sd*bc*sc*i*
alias: pci:v00008086d00001533sv*sd*bc*sc*i*
alias: pci:v00008086d00001F45sv*sd*bc*sc*i*
alias: pci:v00008086d00001F41sv*sd*bc*sc*i*
alias: pci:v00008086d00001F40sv*sd*bc*sc*i*
depends: dca
vermagic: 2.6.33.5-112.fc13.i686.PAE SMP mod_unload 686
parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (max 100000), default 3=adaptive (array of int)
parm: IntMode:Change Interrupt Mode (0=Legacy, 1=MSI, 2=MSI-X), default 2 (array of int)
parm: Node:set the starting node to allocate memory on, default -1 (array of int)
parm: LLIPort:Low Latency Interrupt TCP Port (0-65535), default 0=off (array of int)
parm: LLIPush:Low Latency Interrupt on TCP Push flag (0,1), default 0=off (array of int)
parm: LLISize:Low Latency Interrupt on Packet Size (0-1500), default 0=off (array of int)
parm: RSS:Number of Receive-Side Scaling Descriptor Queues (0-8), default 1, 0=number of cpus (array of int)
parm: VMDQ:Number of Virtual Machine Device Queues: 0-1 = disable, 2-8 enable, default 0 (array of int)
parm: max_vfs:Number of Virtual Functions: 0 = disable, 1-7 enable, default 0 (array of int)
parm: MDD:Malicious Driver Detection (0/1), default 1 = enabled. Only available when max_vfs is greater than 0 (array of int)
parm: QueuePairs:Enable Tx/Rx queue pairs for interrupt handling (0,1), default 1=on (array of int)
parm: EEE:Enable/disable on parts that support the feature (array of int)
parm: DMAC:Disable or set latency for DMA Coalescing ((0=off, 1000-10000(msec), 250, 500 (usec)) (array of int)
parm: LRO:Large Receive Offload (0,1), default 0=off (array of int)
parm: debug:Debug level (0=none, ..., 16=all) (int)