Single session max 1GB - on 10GB link

No. Keep in mind that there are limits depending on Roundtrip time/ping. A network with 80ms ping has a theoretical limit of 1.5Gbit for one TCP Stream. (If the buffers and segment sizes are at default settings)

https://www.switch.ch/network/tools/tcp_throughput/

2 Likes

Arne is right. You might not get a 10 GBit/s link saturated using a single TCP connection. However, the NIC is not helping here either because it further increases latency (because you will have to wait that the first processor core becomes available to handle packets).

You could try a firmware update, another NIC, or replace the whole machine.

Just for reference, our IPFire Enterprise Appliance performs load-balancing over all processor cores and therefore is very power efficient:

The other appliances do this too, which is why they all have relatively small processors because a lot of work is already done by the network controller.

3 Likes

Alright, do you know what supermicro motherboard is use on the appliance?

Thanks

It comes down to more than just the mainboard. There needs to be firmware support as well, the NIC must be able to handle this and of course the software.

ok thanks got it - is there a hardware list if I want to build it my self?

There are some things on the wiki and on here.

I can’t recommend anything else but our own appliances, because those are the only ones that I work with and so much time has gone into them to make features like this working.

1 Like

I understand :slight_smile: - I will re-evaluate the setup

Thanks

1 Like

Apart from the TCP max, this phenomena is also known as an elephant flow. Most hardware vendors struggle with this type of stream, the only way to maximize on speed is to use hardware offloading (e.g. direct switching, bypassing the CPU and security fabric). Anything that the CPU must do (any security inspection) will make the CPU the bottleneck, in which case higher clock speeds will mean better throughput, but processors nowadays have less speed and more cores (which doesn’t help your case).

As said, the way to solve this is to employ parallellism, these flows are typical to backup and research programs, because they tend to move large amounts of data in a single go. Not sure if you actually need the 10gigs, or were just testing things out. :wink:

As mentioned other than some hardware offloading, if relying on CPU interrupts rule of thumb use to be for full duplex throughput was 2 x Interface throughput (bits per second) = CPU speed. Multiple cores have seen other product installation Linux able to tie different interfaces to different cores, depending on hardware. On enterprise hardware, 10G cards have offloading and buffering, etc and are extremely expensive. Dont expect to get anywhere near 10G throughput on lower prices 10G network hardware may interface 10G but throughput is not going to be there and not on most consumer grade workstation, mini PC hardware.

This is indeed very difficult and that is the reason why we are building our own appliances. They are based on Intel designs and use NICs that you can buy from any proper online retailer, but they are carefully selected and tuned. The same NIC on a different board could simply not perform well because of what you just mentioned: CPU clock, memory clock or how PCI lanes are connected and what else is connected to them.

Without spending a lot more money on way too oversized hardware, we believe that our appliances are the best way to run IPFire and get top results with smallest power consumption.

Multiple cores generally do not scale well. A faster core is more efficient, but there is limit to how much performance a single core can provide. We are currently conducting a lot of benchmarks to assess where bottlenecks are and if and how we can solve them, but since our software changes so fast, we didn’t publish anything, yet.

Buffer is great, but it increases latency and nobody likes “bufferbloat”.

1 Like