D525 for 1GbE Internet (IRQ CPU hogging)

I have installed IPFire on a Atom D525 (2C/4T) appliance w Intel 82593V NICs.
I ran a simple speedtest through the IPFire. The appliance bottlenecked at 500 Mbps with the graph indicating that IRQ is the culprit. One core is hogged at 100%.

The typical device looks as following:
04:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
Subsystem: Intel Corporation 82583V Gigabit Network Connection
Flags: bus master, fast devsel, latency 0, IRQ 27
Memory at feae0000 (32-bit, non-prefetchable) [size=128K]
I/O ports at dc00 [size=32]
Memory at feadc000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [e0] Express Endpoint, MSI 00
Capabilities: [a0] MSI-X: Enable- Count=1 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-10-f3-ff-ff-37-37-9a
Kernel driver in use: e1000e
Kernel modules: e1000e
As we can see the MSI is enabled, yet MSI-X isn’t.

ethtool -c red0
Coalesce parameters for red0:
Adaptive RX: n/a TX: n/a
stats-block-usecs: n/a
sample-interval: n/a
pkt-rate-low: n/a
pkt-rate-high: n/a

rx-usecs: 62
rx-frames: n/a
rx-usecs-irq: n/a
rx-frames-irq: n/a

tx-usecs: n/a
tx-frames: n/a
tx-usecs-irq: n/a
tx-frames-irq: n/a

rx-usecs-low: n/a
rx-frame-low: n/a
tx-usecs-low: n/a
tx-frame-low: n/a

rx-usecs-high: n/a
rx-frame-high: n/a
tx-usecs-high: n/a
tx-frame-high: n/a

I’m wondering if anything could be done to allow it to handle 1GbE at native speed?

Increasing buffers from 256 to 4096 marginally improved performance, but not much (ethtool -G <rx / tx> 4096).

Is IPS or QoS enabled? or disabled?

What version of IPFire are you using?

No IPS or QoS. The latest IPFire x64 version.

Update: The built-in irq balancer is poor. It puts all irq load from both active nics onto a single thread. Once I manually tuned SMP affinity to manually assign each NIC IRQ to a different core, I got an instant 50% improvement in throughput (to 750 Mbits).
http://www.alexonlinux.com/smp-affinity-and-proper-interrupt-handling-in-linux

2 Likes