I am currently switching from OPNSense to IPFire because I got very slow network speeds under OPNSense and read that IPFire has no such problems.
Here is a brief overview how my network looks:
In Proxmox:
1.2.3.4 on enp35s0 // internet access
10.10.10.0/24 on vmbr0 // Proxmox LAN to Firewall WAN
10.0.0.0/24 on vmbr1 // Firewall LAN
In IPFire:
10.10.10.1/24 on vmbr0 as RED with gateway to 10.10.10.0
10.0.0.1/24 on vmbr1 as GREEN
VMs use 10.0.0.0/24 network with 10.0.0.1 as gateway/nameserver
Every port is DNATted to 10.10.10.1 except 8006 and 22 for proxmox
To my problem:
VMs don’t have internet access or can ping 10.0.0.1 (destination host unreachable). Didn’t changed anything in IPFire but as I understand should GREEN have internet access.
Also does NAT Routing get blocked. In logs there are many DROP_INPUT’s. Probably because of double NATting and I don’t know how to fix this. double NATting because the logs show that every request goes to 10.10.10.1
What I CAN’T do:
Put RED directly on 1.2.3.4 because I still need access to the proxmox web interface on port 8006 and 22
expose any other MAC address to the outside because my server hoster blocks this
Any help or tip is appreciated. Please ask if you need anything else
If I understand correctly, the VMs in green can’t ping their gateway? Can they ping each other? Can the VM IPFire machine ping the VMs in green? If I were in your position, my base case would be that the proxmox network setting is at fault here. Is it possible that proxmox configuration is the reason your OPNSense is not running at a decent speed?
My suggestion for troubleshooting your system is the following: before installing a firewall like IPFire, you should first install a regular Linux machine in 10.10.10.1 (e.g a Debian, centos or Ubuntu server). This machine should act as a simplified router, configured with the ISP DNS, two virtual interfaces assigned in the same way (vmbr0 and 1), vmbr0 being the internet gateway and on vmbr1 running a DHCP server for 10.0.0.0/24 (of course same Proxmox configuration). Can this simple router get internet access? Can it access the vmbr1-facing machines? Can the VMs in the 10.0.0.0/24 range access their gateway, each other and the ISP internet gateway? If yes, what about speed? Ping and traceroute? Are the logs looking good?
Regardless of your level of understanding of the network plumbing in Linux, you should find plenty of tutorials teaching you how to configure a bare-bone Linux router. This way you would be able to troubleshoot more effectively by removing several layers of complexity (e.g. IPFire networking around “colored” zones, DNS server configuration, the firewall filtering rules and the double NATing situation) and possibly learn a lot about how things work under the hood, which would be very helpful in properly configuring and running a firewall like IPFire.
I was writing a really long answer how it’s not working or why I can’t really just test things as suddently everything works now.
I CHANGED NOTHING
Before you say that something must have been changed, then it was not from my side.
Can’t explain this… just shutted down IPFire this night, booted OPNSense back up so that everything is reachable, shutted OPNSense down and brang IPFire back up to further test… and now it works. I’ve done multiple reboots last nights with no luck so the “power off and on again” trick can’t apply here.
Maybe some cache that had bad infos in it? I don’t know and probably we won’t know
Now I’m feeling really dumb… but at least my firewall works now? I can add rules and they work so… yay? Network speeds ramped up from 300mbit/s behind opnsense to near full gigabit behind ipfire so thats good.
But to answer some questions:
No there is all fine on the proxmox side. Some argue that the performance drop comes from virtualized network interfaces under FreeBSD and the whole packed checksum offload that can’t be done on hardware but I actually don’t know.
This was right. Couldn’t ping the gateway. I didn’t tested each other but I know that IPFire couldn’t also ping the VMs. So both couldn’t reach each other.
But really thank you for advising me how I could have tested this more deeply @cfusco
This does not surprise me one bit. You have a layered and complex system. First, proxmox, an interface to the whole virtual machine stack. Then you have the operating system of the host, then guests, then the router and finally the VMs behind the router. From my point of view it is a miracle that all works most of the time. That’s why “rebooting” sometimes solve the problem. In troubleshooting I always try to reduce the complexity as much as I can and work in negative, by excluding possibilities first. For example, if you remove IPFire and you put a simple bare-bone linux router and the proble persists, you know that your problem has nothing to do with IPFire. Vice-versa, if you solve the problem than you know that it was an issue with IPFire. And so on.
I wish that I could just forward the external IP to IPFire and call it a day but I don’t like the idea of being completly locked out if IPFire breaks. So at least the proxmox ports have to be reachable and thats only possible in this setup.
Probably a second IPv4 would be nice but thats additional money