Hi all,
I've installed FreeBSD 10.2-STABLE on a server with 2*CPU E5-2643v3 (with HyperThreading on) and Intel XL710 4*10G SFP+ card.
At first I updated FreeBSD drivers to 1.4.0 (from download.intel.com).
I see next strange thing - Every acive ixl interface creates 24 queues (6 core *2 (HT) *2 CPU) but uses only 16-17 of them:
irq284: ixl0:q0 164383663 1941
irq285: ixl0:q1 371238730 4384
irq286: ixl0:q2 378286557 4468
irq287: ixl0:q3 365073427 4312
irq288: ixl0:q4 371116376 4383
irq289: ixl0:q5 372589584 4400
irq290: ixl0:q6 361879025 4274
irq291: ixl0:q7 354607200 4188
irq292: ixl0:q8 223602267 2641
irq293: ixl0:q9 199067474 2351
irq294: ixl0:q10 212598000 2511
irq295: ixl0:q11 202534854 2392
irq296: ixl0:q12 212050675 2504
irq297: ixl0:q13 209106917 2469
irq298: ixl0:q14 201452403 2379
irq299: ixl0:q15 203896634 2408
irq300: ixl0:q16 76328643 901
irq301: ixl0:q17 6030 0
irq302: ixl0:q18 5433 0
irq303: ixl0:q19 6804 0
irq304: ixl0:q20 6098 0
irq305: ixl0:q21 6603 0
irq306: ixl0:q22 6476 0
irq307: ixl0:q23 7141 0
irq309: ixl1:q0 161169757 1903
irq310: ixl1:q1 402042077 4748
irq311: ixl1:q2 399166615 4714
irq312: ixl1:q3 389702886 4602
irq313: ixl1:q4 383371508 4528
irq314: ixl1:q5 388621686 4590
irq315: ixl1:q6 385533771 4553
irq316: ixl1:q7 390478220 4612
irq317: ixl1:q8 232313544 2743
irq318: ixl1:q9 248387076 2933
irq319: ixl1:q10 233942388 2763
irq320: ixl1:q11 237794942 2808
irq321: ixl1:q12 227292626 2684
irq322: ixl1:q13 222151566 2623
irq323: ixl1:q14 234209020 2766
irq324: ixl1:q15 217878026 2573
irq325: ixl1:q16 80177041 947
irq326: ixl1:q17 83 0
irq327: ixl1:q18 74 0
irq328: ixl1:q19 201 0
irq329: ixl1:q20 98 0
irq330: ixl1:q21 95 0
irq331: ixl1:q22 91 0
irq332: ixl1:q23 87 0
# top -aSCHP
last pid: 28661; load averages: 7.06, 6.35, 6.23 up 0+23:35:07 17:15:12
391 processes: 31 running, 215 sleeping, 145 waiting
CPU 0: 0.0% user, 0.0% nice, 0.0% system, 39.4% interrupt, 60.6% idle
CPU 1: 0.0% user, 0.0% nice, 0.0% system, 48.8% interrupt, 51.2% idle
CPU 2: 0.0% user, 0.0% nice, 0.0% system, 42.1% interrupt, 57.9% idle
CPU 3: 0.0% user, 0.0% nice, 0.0% system, 40.2% interrupt, 59.8% idle
CPU 4: 0.0% user, 0.0% nice, 0.4% system, 41.3% interrupt, 58.3% idle
CPU 5: 0.0% user, 0.0% nice, 0.0% system, 37.0% interrupt, 63.0% idle
CPU 6: 0.0% user, 0.0% nice, 0.0% system, 35.8% interrupt, 64.2% idle
CPU 7: 0.0% user, 0.0% nice, 0.0% system, 39.0% interrupt, 61.0% idle
CPU 8: 0.0% user, 0.0% nice, 0.0% system, 22.0% interrupt, 78.0% idle
CPU 9: 0.0% user, 0.0% nice, 0.0% system, 26.0% interrupt, 74.0% idle
CPU 10: 0.0% user, 0.0% nice, 0.0% system, 17.7% interrupt, 82.3% idle
CPU 11: 0.0% user, 0.0% nice, 0.0% system, 19.3% interrupt, 80.7% idle
CPU 12: 0.0% user, 0.0% nice, 0.4% system, 25.2% interrupt, 74.4% idle
CPU 13: 0.0% user, 0.0% nice, 0.0% system, 23.6% interrupt, 76.4% idle
CPU 14: 0.0% user, 0.0% nice, 0.0% system, 22.4% interrupt, 77.6% idle
CPU 15: 0.0% user, 0.0% nice, 0.0% system, 26.8% interrupt, 73.2% idle
CPU 16: 0.0% user, 0.0% nice, 1.2% system, 1.6% interrupt, 97.2% idle
CPU 17: 0.0% user, 0.0% nice, 0.8% system, 0.0% interrupt, 99.2% idle
CPU 18: 0.0% user, 0.0% nice, 1.2% system, 0.0% interrupt, 98.8% idle
CPU 19: 0.0% user, 0.0% nice, 0.4% system, 0.0% interrupt, 99.6% idle
CPU 20: 0.0% user, 0.0% nice, 3.5% system, 0.0% interrupt, 96.5% idle
CPU 21: 0.0% user, 0.0% nice, 1.6% system, 0.0% interrupt, 98.4% idle
CPU 22: 0.0% user, 0.0% nice, 0.8% system, 0.0% interrupt, 99.2% idle
CPU 23: 0.0% user, 0.0% nice, 2.0% system, 0.0% interrupt, 98.0% idle
# netstat -I ixl0 -w1 -h
input ixl0 output
packets errs idrops bytes packets errs bytes colls
235K 0 0 126M 300K 0 321M 0
233K 0 0 114M 297K 0 312M 0
232K 0 0 116M 300K 0 315M 0
227K 0 0 108M 297K 0 316M 0
Where is the source of problem that network adapter doesn't utilize all queues ?
Clik here to view.