Progress on using a Raspberry Pi 4 (compute module)

Hardware Single Board Computers
1

Hi,

This post summarizes my experience setting up a Raspberry Pi Compute Module 4 with IpFire, specifically a Raspberry Pi compute module 4 with a DFRobot Router Carrier Board Mini. I thought it might be helpful to share this experience, in case anyone else is interested in a similar set up.

This carrier board has a PCIe attached second gigabit NIC, and various video reviews indicate that it can saturate the gigabit NICs for NAT.

It wasn’t completely clear if IpFire is supported with the latest Raspberry Pi 4 boards. The good news is that this appears to be the case with the new board I purchased a week ago (yes I managed to purchase one, finally).

The installation on a compute module is a bit tricky, especially with an eMMC flash version (which is what I bought). In order to get the image onto the eMMC flash, it’s necessary to install the board onto the separate I/O board and set the J2 jumper in order that the onboard storage is exposed as a flash drive. This thus requires that the board is physically installed onto the I/O board, removed, then installed again onto the DFRobot board. The other problem I found was that I couldn’t get my PC to connect to the board via the Raspberry Pi usbboot tool, and so I ended up setting up a separate Raspberry Pi 4 (that I already had) in order to write the image to the compute module 4 storage.

I would thus advise not bothering with the onboard storage version, and just go for a lite compute module, along with a separate micro SD card (which can then be imaged separately), as the DFRobot carrier board has a micro SD slot.

The other gotcha I discovered is that the USB ports are disabled by default, so a bit of config is required to be added to config.txt:

dtoverlay=dwc2,dr_mode=host
org_mode=1

After that, I was then able to configure IpFire with a keyboard/HDMI connected monitor, with the compute module still mounted onto the I/O board. As the I/O board only has a single NIC, I did not configure the RED interface. I then enabled permanent SSH access via the web interface, in order to configure the RED interface via SSH.

Once finished, I removed the compute module from the I/O board, mounted it onto the DFRobot dual NIC board, logged in via SSH, configured the RED interface, and then disabled SSH access.

Once that was done, everything worked great, apart from one thing, that may or may not be an issue depending on your line speed.

So after finishing the installation, I did some performance testing with a gigabit WAN link. The first test was just with speedtest. The second test was with iperf3 between a client on the GREEN side and a PC set up as another router on the RED side.

With the initial set up, I only got about 250 Mbps, both with speedtest and with iperf3. I did some investigation, and found (via top) that the NICs were saturating one core of the CPU with software interrupts. After configuring the NIC interrupt affinity to be on different cores, I managed to get about 500 Mbps with speedtest and iperf3.

I was thus a bit puzzled, as the video reviews I had watched had indicated that this set up should saturate the NICs when used as a NAT router. So finally I decided to investigate further, by wiping the IpFire installation, installing a Linux distribution, and configuring an iptables masquarade manually.

With this set up, I was able to get consistent 930-950 Mbps.

So in conclusion, it looks like IpFire has some performance issue, compared with a basic Linux installation with masquerading. I made sure that all services were turned off from within the UI, no additional firewall rules were defined, and intrusion prevention / blocklists were turned off.

I guess this performance loss doesn’t matter for slower WAN links, but I thought I would nevertheless finish this post with a question: what is the nature of this drop in performance? It would be great to be able to use this hardware set up with a Gb WAN speed that can use all the bandwidth.

3 Likes
2

try to enable the same default and minimal statefull rules present in IPFire, in iptables of the reference standard linux distribution and likely you will get similar numbers. A statefull firewall is expensive.

iptables -L -nvx

you can see the rules with this command

1 Like
3

So does the software interrupt cpu time reported by top contain the actual processing cpu time of the firewall rules, and thus represents the additional work done as represented by the iptables rules?

4

Not an expert, but I expect as much. I searched for confirmation and I found this link, claiming a substantial increase of CPU use in statefull inspection.

5

That was a very interesting read. Thank you @cfusco! And thanks for the writeup on the RPi4 too.

6

To conclude on this investigation, I would suggest that the Raspberry Pi 4 should be fine for WAN link speeds up to about 100Mbs or so, but above that, the cpu cores are too slow to process the statefull inspection. Perhaps tweaking the NIC interrupt cpu affinity could help somewhat, but the best I found was a doubling of throughput.

Due to this, possibly one might as well use a standard Pi 4 with a USB attached second NIC. This would avoid the installation complications required if using the compute module. I haven’t tried this, so cannot comment on the reliability of a USB NIC dongle. I can say that the DFRobot board has been working very reliably.

However, further thought led me to conclude that a better way of avoiding the manual replugging of the compute module between the I/O board and the DFRobot board would be to use a standard Pi 4 to install and configure IpFire onto an sd card, configure the GREEN interface, enable SSH, then move the sd card to a compute module lite / DFRobot combination and reconfigure the RED interface via SSH. This would necessitate the purchase of a second Pi 4 board, but would avoid the need to purchase the compute module I/O board (which costs a similar amount). The CM4/DFRobot + case set up is very nice.

One additional thing: I forgot to add that I also updated the EEPROM before installing. This is discussed here: Need to full support Raspberrypi 4B! - #4 by arne_f

1 Like
7

Nope. I’ve both. RPi4B 4GB and CM4 8/32GB and never had any limitation with Vodafone CableMax 1Gbit.

8

Is this running IpFire? As mentioned in my first post, I can saturate the Gb NICs with just Linux masquerading, but the throughput drops to 250Mbs with IpFire.

9

Yes. When the support came out, IPFire moved straight to the devices and I did speedtests to see if they may be a bottleneck but they weren’t. Can’t test it anymore because I found out for myself that a 4G/LTE flat is enougn and I don’t need a 1GBit WAN (wasting money and energy) so I canceled it.

Edit: can’t test it is not true. I don’t need a 1GBit WAN, just take a PC running a iperf server and connect it to IPFire RED and run a iperf client and test the throughput.