I found out what happened here. Prior to the NanoPi R6S, I had used a Fortigate 61E firewall and prior to that an Edgerouter Lite. I rechecked this circuit with speedtest from the Edgerouter Lite but hadn’t done them on the Fortigate. The Edgerouter Lite maintains an A rating but the Fortigate gets an F rating – neither have QoS enabled. So both the Nanopi R6S and the Edgerouter Lite can get an A without Cake QoS but the Fortigate fails miserably.
You named a bunch of devices with no context. So which device is IPFire installed on, and does it have a direct path to the ISP gateway?
so you went from a half way good RISC proccesor to a somewhat adequate ASIC system. Fortigate uses ASiC based architecture which does not have concurrency so it has to divide functions across asic co-processors that have a fraction of the throughput the main chip has.
The edge router and the nanopi are better engineered.
The Edgerouter Lite is a device from 2013 that was less than $100 even then. The Forgitate 61E is technically still in support until next year and has 1Gbs IPSEC VPN capabilities. It was also priced at 10x the Edgerouter and contains a stateful firewall. It was given to me as a hand-me-down from work during Covid.
We’ll have to respectfully disagree on which is better engineered. The Edgerouter’s simplicity has given it an edge in uniform latency. The Nanopi R6S was a recent purchase to allow for CAKE QoS at 1Gbs while still allowing for a stateful firewall.
I was looking at replacing it with an IPFire device but then noticed that even without QoS enabled, it was doing OK in bufferbloat tests. The Edgerouter Lite is too. So I’m reevaluating the need for QoS at this point.
Theoretically, if the WAN connection can service traffic without interruptions to and from the media change device, the limitation will be how well the media change device conveys on the isp media (i.e., cable, fiber).
The fortigate was probably fine for you because you probably didn’t have WAN net traffic greater than 1.5Gb.
So in reality QoS is mainly for error free sustained transfer rate and will sacrifice bandwidth to achieve this. However, its just tuning Tx and Rx buffers and once buffers are complementary between each other, QoS isn’t needed any more because changing them would reduce the bandwidth even further. I want to try it because I haven’t done it in a decade or so, but I know its not needed on modern broadband. And modern drivers and OS have buffer scaling built in so QoS becomes redundant.
Both the ONT and the Fortigate 61E only have 1Gbs Ethernet ports so the hard limit is 1Gbs – not 1.5Gbs. It’s also always below that due to overhead so more like 930-940Mbs. The NanoPi and the Edgerouter Lite have the same limitation. What was happening on the Fortigate was probably additional buffers on the device used for traffic inspection and potentially just to keep the ASIC fed with data but that would have the unfortunate effect of delaying traffic more than if the buffer didn’t exist. I’m not a computer engineer but that makes more sense to me.
The Fortigate will probably just sit on a shelf until I decide to throw it away so it’s a moot point unless there’s an easy way of tuning it to an A bufferbloat rating. I’ll use the NanoPi R6S as my primary device and the Edgerouter Lite will be my backup.
I think most of the time those bufferbloat tests give out false readings. Because the results are not exactly consistent. The Fortigate is ok, I just touched on some of the engineering setbacks of that processor technology. The main Asics processors typically have 300-700Gb of throughput, but the co-processors throughput are 1.5-3Gb. Which definitely you can build a nice performing 1Gb router. 930-940 Mb on fiber would be the correct bandwidth results for 1Gb fiber with PPPoe. Software PPPoe induces that 50-60Mb overhead. Hardware accelerated PPPoe is about 1/2 that.
No PPPoE on the router here.
what kind of connection is it? Does it run in a VLAN?
No VLAN. It’s a normal Ethernet connection handoff to the ONT. Any details of how they do things beyond it are hidden from the customer.