ONT fiber connection (Telekom)

Networking
1

Hello,

We have upgraded our internet connection from VDSL 100/40 to fiber optic 1000/500.
I use a PC Engines APU2D3 with I210 Gigabit Network adapters as a router.

I had set up our old connection via the VDSL interface with a modem. So far, so good.

Now I have an ONT Glasfaser Modem 2 kaufen | Telekom

And I need VLAN 7 to establish the connection, so the connection works with the VDSL interface. However, I only get a download speed of 170 Mbps and an upload speed of 500 Mbps.

IPS is disabled… but it makes no difference to the download.

Establishing a connection via the PPPoE interface doesn’t work at all.

So I used a cheap router to check if it was my connection, but I get 980/480 Mbps with it.

Am I doing something wrong with the connection setup? Is my hardware not good enough for downloads above 170 Mbps? Why is only the download affected and not the upload?

2

Try disabling Qos?

3

was not enabled :frowning:

4

“VDSL” is “PPPoE via VLAN”. It is missnamed because the German Telecom has used this long time only for VDSL lines.

PPPoE made some CPU load and can only use one CPU core. Im not sure how much the APU can handle with PPPoE.

5

lets look at some things.I never preformed this net config sanity check in IPFire as I was going to save this when I go through helping revising networking.

execute this and post the results:

awk '{for (i=1; i<=NF; i++) printf strtonum("0x" $i) (i==NF?"\n":" ")}' /proc/net/softnet_stat

the interrupts

grep -E "CPU|red0" /proc/interrupts

netdev_budjets:

sysctl net.core.netdev_budget_usecs net.core.netdev_budget

lets see what cstate mode driver is used:

cat /sys/devices/system/cpu/cpuidle/current_driver

if intel_idle driver is used, what max cstate is set:

cat /sys/module/intel_idle/parameters/max_cstate

TCP system socket buffers

sysctl net.ipv4.tcp_rmem
sysctl net.ipv4.tcp_wmem

UDP buffers

sysctl net.core.rmem_max
sysctl net.core.wmem_max

is pruning or collapsing tcp going on?:

nstat -az TcpExtTCPRcvCollapsed TcpExtRcvPruned

tcp listen backblock:

sysctl net.core.somaxconn

UDP GRO status

ethtool -k red0 | grep generic-receive-offload

health status of red0

ethtool -S red0

setting of timestamps subroutine that prevents remote fingerprinting and certain time based attacks (this should be a value of 1)

sysctl net.ipv4.tcp_timestamps

does red0 have pause frames support enabled.

ethtool --show-pause red0
1 Like
6 
# awk '{for (i=1; i<=NF; i++) printf strtonum("0x" $i) (i==NF?"\n":" ")}' /proc/net/softnet_stat
100225973 0 283515 0 0 0 0 0 0 0 0 0 0 0 0
4962992 0 8389 0 0 0 0 0 0 0 0 0 1 0 0
6568104 0 9605 0 0 0 0 0 0 0 0 0 2 0 0
9218254 0 23286 0 0 0 0 0 0 0 0 0 3 0 0

# grep -E "CPU|red0" /proc/interrupts
            CPU0       CPU1       CPU2       CPU3       
  45:          0          0          0          2  PCI-MSIX-0000:02:00.0    0-edge      red0
  46:   21087574          0          0          0  PCI-MSIX-0000:02:00.0    1-edge      red0-TxRx-0
  47:          0    2501612          0          0  PCI-MSIX-0000:02:00.0    2-edge      red0-TxRx-1
  48:          0          0    2906279          0  PCI-MSIX-0000:02:00.0    3-edge      red0-TxRx-2
  49:          0          0          0    6519143  PCI-MSIX-0000:02:00.0    4-edge      red0-TxRx-3

# sysctl net.core.netdev_budget_usecs net.core.netdev_budget
net.core.netdev_budget_usecs = 2000
net.core.netdev_budget = 300

# cat /sys/devices/system/cpu/cpuidle/current_driver
acpi_idle

# cat /sys/module/intel_idle/parameters/max_cstate
9

# sysctl net.ipv4.tcp_rmem && sysctl net.ipv4.tcp_wmem
net.ipv4.tcp_rmem = 4096	87380	16777216
net.ipv4.tcp_wmem = 4096	16384	16777216

# sysctl net.core.rmem_max && sysctl net.core.wmem_max    
net.core.rmem_max = 212992
net.core.wmem_max = 212992

# nstat -az TcpExtTCPRcvCollapsed TcpExtRcvPruned
#kernel
TcpExtRcvPruned                 0                  0.0
TcpExtTCPRcvCollapsed           0                  0.0

# sysctl net.core.somaxconn
net.core.somaxconn = 4096

# ethtool -k red0 | grep generic-receive-offload
generic-receive-offload: on

# ethtool -S red0
NIC statistics:
     rx_packets: 31936973
     tx_packets: 34316097
     rx_bytes: 19542637036
     tx_bytes: 36623645811
     rx_broadcast: 0
     tx_broadcast: 17
     rx_multicast: 0
     tx_multicast: 0
     multicast: 0
     collisions: 0
     rx_crc_errors: 0
     rx_no_buffer_count: 38954
     rx_missed_errors: 247445
     tx_aborted_errors: 0
     tx_carrier_errors: 0
     tx_window_errors: 0
     tx_abort_late_coll: 0
     tx_deferred_ok: 0
     tx_single_coll_ok: 0
     tx_multi_coll_ok: 0
     tx_timeout_count: 0
     rx_long_length_errors: 0
     rx_short_length_errors: 0
     rx_align_errors: 0
     tx_tcp_seg_good: 0
     tx_tcp_seg_failed: 0
     rx_flow_control_xon: 0
     rx_flow_control_xoff: 0
     tx_flow_control_xon: 0
     tx_flow_control_xoff: 0
     rx_long_byte_count: 19542637036
     tx_dma_out_of_sync: 0
     tx_smbus: 0
     rx_smbus: 0
     dropped_smbus: 0
     os2bmc_rx_by_bmc: 0
     os2bmc_tx_by_bmc: 0
     os2bmc_tx_by_host: 0
     os2bmc_rx_by_host: 0
     tx_hwtstamp_timeouts: 0
     tx_hwtstamp_skipped: 0
     rx_hwtstamp_cleared: 0
     rx_errors: 0
     tx_errors: 0
     tx_dropped: 0
     rx_length_errors: 0
     rx_over_errors: 0
     rx_frame_errors: 0
     rx_fifo_errors: 247445
     tx_fifo_errors: 0
     tx_heartbeat_errors: 0
     tx_queue_0_packets: 12887844
     tx_queue_0_bytes: 15065102326
     tx_queue_0_restart: 2708
     tx_queue_1_packets: 4322497
     tx_queue_1_bytes: 3800259516
     tx_queue_1_restart: 77
     tx_queue_2_packets: 5399564
     tx_queue_2_bytes: 4884339680
     tx_queue_2_restart: 32
     tx_queue_3_packets: 11706192
     tx_queue_3_bytes: 12599059199
     tx_queue_3_restart: 82
     rx_queue_0_packets: 31936973
     rx_queue_0_bytes: 19287535898
     rx_queue_0_drops: 0
     rx_queue_0_csum_err: 0
     rx_queue_0_alloc_failed: 0
     rx_queue_1_packets: 0
     rx_queue_1_bytes: 0
     rx_queue_1_drops: 0
     rx_queue_1_csum_err: 0
     rx_queue_1_alloc_failed: 0
     rx_queue_2_packets: 0
     rx_queue_2_bytes: 0
     rx_queue_2_drops: 0
     rx_queue_2_csum_err: 0
     rx_queue_2_alloc_failed: 0
     rx_queue_3_packets: 0
     rx_queue_3_bytes: 0
     rx_queue_3_drops: 0
     rx_queue_3_csum_err: 0
     rx_queue_3_alloc_failed: 0

# sysctl net.ipv4.tcp_timestamps
net.ipv4.tcp_timestamps = 1

# ethtool --show-pause red0
Pause parameters for red0:
Autonegotiate:	on
RX:		on
TX:		off
7

The third column here is dropped frames from socket connection.

net.core.netdev_max_backlog is set at the default value of 1000.

To find the correct value for this, write down your numbers, then you create the new parameter in a file then apply it and watch for the counter to stop increasing the numbers in the 3rd column.

We will look at some other things, flow control is a big factor, tx_queue_0_restart is a big indicator that flow control needs some attention because TX ring buffer is full and had to delay packets which logs in the tx_queue_0_restart counter.

EDITED After reviewing:

I do see some revisions of the stack is needed for more than 2gb throughput and all power states disabled, but the driver isn’t setting flow control properly. You see the driver for these have been incorrect for a long time and wasn’t fixed until around kernel 6.10 when they fixed the dual port version of this chip in the same driver.

The output should look like this:

[root@pwrtower ~]#  ethtool --show-pause red0
Pause parameters for red0:
Autonegotiate:	on
RX:		on
TX:		on
RX negotiated: on
TX negotiated: on

and the driver should be in this mode:

ip link show red0
2: red0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 1c:86:0e:21:3f:f1 brd ff:ff:ff:ff:ff:ff

EDIT

what are you running as ipfire?
type this at your terminal to find out:

uname -a

even though one of the parameters I know need to be evaluated. Which is PREEMPT_RT vs. PREEMPT vs. PREEMPT_DYNAMIC.

8 
# uname -a
Linux router.localdomain 6.12.13-ipfire #1 SMP PREEMPT_DYNAMIC Thu Feb 20 03:36:32 GMT 2025 x86_64 GNU/Linux

just a shot in the dark

# ethtool -A red0 autoneg off rx on tx on
# ethtool --show-pause red0
Pause parameters for red0:
Autonegotiate:	off
RX:		on
TX:		on

same result.

I also have an APU with the i211 chip set, which is actually the cheaper chip.

but same driver …

So at the end off the day i need something with better chip set or more power, if i want to upgrade my download speed.

9

That explains a lot to me.
You see, the I210 and the I220 have four buffer registers and the 211 only has two. So to get the same performance out of it would require changing a couple of things in the stack so you have the same queue size the driver wants to run with.

Lets try first:

sysctl -w net.core.netdev_max_backlog=4000

and see if this gets better, if so, add that line to /etc/sysconfig/rc.local file

Then set tx queue to a higher payload value greater than backlog for proper RX queue scaling

ip link set red0 txqueuelen 5000

see if this gets better, if so, add that line to /etc/sysconfig/rc.local file

Theoretically this will set memory buffers to compensate where the hardware doesn’t have them. No one in Linux Main had one of these interfaces to play with so none of us knew how this driver was going to act with the hardware.

10

:rofl:

https://community.ipfire.org/search?expanded=true&q=mini%20pppoe

welcome to the futurepast.
dont trash your apu though!
once the isps in digiland
get rid of pppoe it will perform :mechanical_arm:
if you like you can dig deeper about the
ancient-pppoe-limitations via a searchgengine :detective:

11

well 980/480 sounds like a 1000/500 fiber with PPPoe overhead.

I would just plug in a usb ethernet adapter because the driver for the i210 is not very good for WAN use.

1 Like
12

Hmm… USB NIC (RTL8153) is a game changer.

910/420…

It does get quite warm though :slight_smile:

At the end of the day, I’ll probably get some more fashionable hardware for the WAN. I’ve already considered an n150 @6x i226v, as 3 more ports are definitely useful for me. I don’t know what I want to do with all that computing power, but I’ll figure something out. Thanks for the suggestion for a USB NIC. I was already looking into getting a PCIe adapter and hadn’t thought of the most obvious thing, which was already in my drawer.

13

Well if you get a PCI card, I would suggest getting a TrendNET TEG-25GECTX because that was the card the realtek driver was worked out on and my team did a great job on that driver. The rest of the bandwidth is small mismatched buffer sizes I would have to guess.

Problem has always been applications like this require tuning of the hardware. Which is why for several years there wasn’t many of these router OS systems made or gained a lot of popularity until privacy concerns surfaced. What complicates it is you have to support everyone’s ethernet interface as well as develop mechanisms to tune the hardware that is attached or tune it where most interfaces will work well. Currently, no one has something like that and as a result, some fair better than others across the different ones with a generic stack tuning.

14

:thinking:
APU2D3+RTL8153+ipfire+PPPoE=910/420 Mbit/s
can you|one post a link for APU2D3 docs :magnifying_glass_tilted_right: